Shop OBEX P1 Docs P2 Docs Learn Events
OT: Linux Web hosting. — Parallax Forums

OT: Linux Web hosting.

Martin HodgeMartin Hodge Posts: 1,246
edited 2013-03-14 22:38 in General Discussion
I can remember back when I had a small web hosting business I would start to panic if the load avg got above 5.00. So I find it disconcerting when I SSH into my account at Bluehost and find this...
top - 10:27:52 up 58 days, 13:46,  1 user,  load average: 85.44, 66.84, 52.65
Tasks:   4 total,   1 running,   3 sleeping,   0 stopped,   0 zombie
Cpu(s): 73.4%us, 16.6%sy,  0.0%ni,  7.2%id,  2.5%wa,  0.0%hi,  0.4%si,  0.0%st
Mem:  33002552k total, 26259892k used,  6742660k free,  1341120k buffers
Swap:  8388600k total,   201644k used,  8186956k free, 13008872k cached


  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
15831 mghdesig  20   0 14900 1188  952 R  0.3  0.0   0:00.08 top
 7611 mghdesig  20   0 11472 1712 1328 S  0.0  0.0   0:00.02 bash
17012 mghdesig  20   0  260m  24m 8972 S  0.0  0.1   0:00.18 fcgiphp5
30394 mghdesig  20   0 15992 1768 1432 S  0.0  0.0   0:00.04 imap

I've never seen it below 40! Of course when I bring this up with support I get the standard canned "Optimize your website" reply.

Comments

  • Oldbitcollector (Jeff)Oldbitcollector (Jeff) Posts: 8,091
    edited 2013-03-11 09:26
    Are they sharing your server with other accounts? Sounds like this box is loaded up pretty heavy.

    Jeff
  • Heater.Heater. Posts: 21,230
    edited 2013-03-11 10:19
    That load figure does not tell us much unless we know how many requests per second you are handling at the time.
    And what do those requests have to do? What hardware is this? Is it a virtual machine? etc etc.
  • Martin HodgeMartin Hodge Posts: 1,246
    edited 2013-03-11 11:07
    Oh yes, it's a shared box. I'm just shocked they would let it get this bad.
  • softconsoftcon Posts: 217
    edited 2013-03-12 14:34
    Hard to tell from mem info statements, but it looks to me like the box only has 32MB of ram, and that's a disaster for any server, especially one hosting multiple domains. When I had my linux hosts running (I outsource it now, it's cheaper) I never had less than 256MB in any of my machines, and often enough, that was the minimum I had, usually more. I tried to keep the system load around 50% max load, which wasn't hard, even with folks hosting games on the server. they almost never went above 30 percent usage or so, though once, before I upgraded the memory, it was hammering the swap space pretty hard, which does significantly raise the cpu usage. After the upgrade (doubled the memory) a gcc compile went from around two hours to only about 25 minutes or so, so those that say memory doesn't matter obviously never benchmarked their own systems.
    Just 4 processes on the machine? Not with that kind of a cpu load. <sigh> Sometimes, I wonder what the heck modern day isps/web hosts teach their hardware folks (if they teach them anything at all) I find that calling a web host with questions on anything but billing meets with a lot of blaming and flat out wrong answers. So, for my sites these days, I just signed up as a reseller with a relatively inexpensive company (not godaddy) and resell web hosting to myself, it's cheaper, and I get unlimited disk space, mysql databases, bandwidth, and email addresses, and it's all for less than it'd cost me to maintain my own servers, so that makes me happy too. :)
  • Oldbitcollector (Jeff)Oldbitcollector (Jeff) Posts: 8,091
    edited 2013-03-12 15:01
    @softcon,

    Doing the exact same thing here.. totally cheaper to do this..

    Funny part was a few years ago when I opened my shop in Downtown Orrville, the local cable company wouldn't put cable internet into my office without sending down their office folks to check to make sure that I wasn't setting up servers, Portmasters, etc. in violation of their contract. I laughed when they came down to look over my shop and give me a contract to sign. It's simply not economical to host your own stuff any more.

    Jeff
  • Ding-BattyDing-Batty Posts: 302
    edited 2013-03-12 16:13
    softcon wrote: »
    Hard to tell from mem info statements, but it looks to me like the box only has 32MB of ram, and that's a disaster for any server, especially one hosting multiple domains.

    I beg to differ -- I see "Mem: 33002552k total" and I read that number as 33,002,552 * 1024 bytes, or 32GB, not 32MB.
  • Martin HodgeMartin Hodge Posts: 1,246
    edited 2013-03-12 16:30
    Ah, had the CPUs hidden:
    top - 17:29:04 up 60 days, 20:47,  1 user,  load average: 17.62, 16.15, 20.76
    Tasks:   4 total,   1 running,   3 sleeping,   0 stopped,   0 zombie
    Cpu0  : 47.8%us, 14.9%sy, 16.0%ni, 15.5%id,  5.5%wa,  0.0%hi,  0.2%si,  0.0%st
    Cpu1  : 51.0%us, 14.7%sy, 14.5%ni, 16.3%id,  3.4%wa,  0.0%hi,  0.2%si,  0.0%st
    Cpu2  : 48.7%us, 14.4%sy, 16.3%ni, 18.5%id,  2.1%wa,  0.0%hi,  0.0%si,  0.0%st
    Cpu3  : 47.0%us, 14.6%sy, 15.1%ni, 19.4%id,  3.8%wa,  0.0%hi,  0.1%si,  0.0%st
    Cpu4  : 47.3%us, 14.0%sy, 16.2%ni, 19.1%id,  3.3%wa,  0.0%hi,  0.1%si,  0.0%st
    Cpu5  : 44.7%us, 14.6%sy, 16.3%ni, 22.4%id,  2.0%wa,  0.0%hi,  0.0%si,  0.0%st
    Cpu6  : 51.0%us, 13.8%sy, 18.4%ni, 15.8%id,  0.9%wa,  0.0%hi,  0.0%si,  0.0%st
    Cpu7  : 55.7%us, 11.3%sy, 18.5%ni, 14.2%id,  0.3%wa,  0.0%hi,  0.0%si,  0.0%st
    Cpu8  : 47.5%us, 13.5%sy, 16.1%ni, 18.5%id,  4.3%wa,  0.0%hi,  0.0%si,  0.0%st
    Cpu9  : 44.6%us, 13.7%sy, 10.8%ni, 22.4%id,  3.6%wa,  0.0%hi,  5.0%si,  0.0%st
    Cpu10 : 42.9%us, 14.0%sy, 14.5%ni, 26.3%id,  1.9%wa,  0.0%hi,  0.4%si,  0.0%st
    Cpu11 : 42.0%us, 14.2%sy, 13.8%ni, 22.7%id,  6.7%wa,  0.1%hi,  0.4%si,  0.0%st
    Cpu12 : 42.5%us, 13.6%sy, 14.4%ni, 26.0%id,  3.3%wa,  0.0%hi,  0.1%si,  0.0%st
    Cpu13 : 40.8%us, 13.5%sy, 16.1%ni, 27.8%id,  1.8%wa,  0.0%hi,  0.0%si,  0.0%st
    Cpu14 : 51.2%us, 11.6%sy, 18.4%ni, 18.1%id,  0.6%wa,  0.0%hi,  0.0%si,  0.0%st
    Cpu15 : 57.1%us, 10.3%sy, 19.0%ni, 13.4%id,  0.2%wa,  0.0%hi,  0.0%si,  0.0%st
    Mem:  33002552k total, 25463088k used,  7539464k free,  1787240k buffers
    Swap:  8388600k total,   175072k used,  8213528k free, 14558756k cached
    
    
      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                
     6891 mghdesig  20   0  277m  41m 9380 S  0.0  0.1   0:00.84 fcgiphp5                                                                                                               
     7331 mghdesig  20   0 11472 1708 1328 S  0.0  0.0   0:00.00 bash                                                                                                                   
     7601 mghdesig  20   0 14900 1192  956 R  0.0  0.0   0:00.03 top                                                                                                                    
     9128 mghdesig  20   0 15832 1488 1240 S  0.0  0.0   0:00.05 imap
    

    So 16 cores of....
    processor       : 0
    vendor_id       : GenuineIntel
    cpu family      : 6
    model           : 26
    model name      : Intel(R) Xeon(R) CPU           E5530  @ 2.40GHz
    stepping        : 5
    cpu MHz         : 2400.068
    cache size      : 8192 KB
    physical id     : 0
    siblings        : 8
    core id         : 0
    cpu cores       : 4
    apicid          : 0
    initial apicid  : 0
    fpu             : yes
    fpu_exception   : yes
    cpuid level     : 11
    wp              : yes
    flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
    bogomips        : 4800.13
    clflush size    : 64
    cache_alignment : 64
    address sizes   : 40 bits physical, 48 bits virtual
    power management:
    

    Looking better ATM, but I wonder how many thousand sites are on this server.
  • Martin HodgeMartin Hodge Posts: 1,246
    edited 2013-03-12 16:34
     [~]# cat /proc/meminfo
    MemTotal:       33002552 kB
    MemFree:         7702024 kB
    Buffers:          253072 kB
    Cached:         15100368 kB
    SwapCached:        59852 kB
    Active:         17082020 kB
    Inactive:        6176820 kB
    Active(anon):    7099872 kB
    Inactive(anon):  1648564 kB
    Active(file):    9982148 kB
    Inactive(file):  4528256 kB
    Unevictable:      183772 kB
    Mlocked:          183772 kB
    SwapTotal:       8388600 kB
    SwapFree:        8213528 kB
    Dirty:             24456 kB
    Writeback:             0 kB
    AnonPages:       8035536 kB
    Mapped:            71944 kB
    Shmem:            835304 kB
    Slab:            1518680 kB
    SReclaimable:    1214436 kB
    SUnreclaim:       304244 kB
    KernelStack:       14632 kB
    PageTables:       161440 kB
    NFS_Unstable:          0 kB
    Bounce:                0 kB
    WritebackTmp:          0 kB
    CommitLimit:    24889876 kB
    Committed_AS:   29161612 kB
    VmallocTotal:   34359738367 kB
    VmallocUsed:      350372 kB
    VmallocChunk:   34341895192 kB
    HardwareCorrupted:     0 kB
    HugePages_Total:       0
    HugePages_Free:        0
    HugePages_Rsvd:        0
    HugePages_Surp:        0
    Hugepagesize:       2048 kB
    DirectMap4k:        7680 kB
    DirectMap2M:    33538048 kB
    

    And 34TB of virtual memory? Could that be right?
  • pedwardpedward Posts: 1,642
    edited 2013-03-12 22:47
    That company is screwing the pooch, and I wish I could say it the impolite way!

    On MPU Linux boxes, a LA of 1 per core is equivalent to 100% utilization. They've got 2 CPUs which are 4 core with HT.

    A LA of 85 / 16 = 5

    I have a 4 core HT Core i7 machine at work, and I can say that HT helps, but it isn't an extra core, it's a lot like Chip's multi-threading in the P2.

    Overall, that machine is grossly overloaded, well and truly beyond accepted best practices.

    My guess is that they have a few thousand shared users on that machine and they don't implement any sort of fair share queuing. They would be better off to segregate the machine into 8 or 16 XEN VMs and balance the users across the VMs, then use the ban hammer on the abusive customers.

    They only have 8GB of swap on that machine, the 34TB is probably the virtual address limit of the CPU--40 bits physical, 48 bits virtual
  • Heater.Heater. Posts: 21,230
    edited 2013-03-13 00:34
    Martin,
    And 34TB of virtual memory? Could that be right?

    On a sixty four bit machine you have a 40 bit physical memory address, ie that many pins on the address bus. That is enough to access one terabyte of real RAM if you had it.
    As shown above the internal virtual address is 48 bit, enough for 256 terabytes.

    The key here is "virtual", Your processor can access that address range but it may not be real memory. When an access like that fails the OS can map it to swap space on disk.
  • Martin HodgeMartin Hodge Posts: 1,246
    edited 2013-03-13 01:52
    Right. "Swap" was what I was thinking of in the first place. At any rate, I'm browsing some VPS services to eventually migrate to.
  • softconsoftcon Posts: 217
    edited 2013-03-13 07:20
    When we got our current isp, I asked them what it would cost for a static ip, just out of curiosity, and they said it's not offered to residential customers, but if I wanted a business account, I could get a static ip for $400 a month. I laughed outright, and explained that for $400 a month, I could get one from icann for that price.
    Turns out I'd remembered wrong, icann charges (last time I checked anyway) about 10K a year for a block of ips, but the service I use for web hosting offers static ips for $2 a month, so no contest there.
    It is absolutely mind boggling the things local isps believe they can charge for services.
    I'd swear they all believe that nobody using their services has a clue about networking or actual cost of services provided.
    It's quite sad really.
    Then again, most folks in this town use a service that costs 60 or 70 bucks a month for 1.5 dsl, and for less than that this company provides 10MBPS service, so I guess they're somewhat justified in their views, but still, it rankles to be treated like a total ignorant savage when it comes to technology. *grumble*
  • Heater.Heater. Posts: 21,230
    edited 2013-03-13 08:51
    Given that we ran out of IPv4 addresses some time ago it's not surprising that they charge an arm and a leg for one. In fact I think the idea is that they really don't want you to have one at all. As a residential customer they don't cater for you wanting to run servers. If you are not running services then you don't need a static IP address.

    All of which is very annoying as I like to be able to log into my home machine from outside sometimes. You can get around not having a static IP by using a dynamic DNS services.

    Of course the next step is that you don't even get a public IP address so you can't reach your home machine at all without it creating a VPN tunnel to some external server.

    Hopefully IPv6 will come soon and save us from all that nonsense. I'm just afraid the providers will find some way to bugger that up as well.

    Amazingly I found that a couple of years ago every computer in Helsinki university had a public IP address. What luxury.
  • RDL2004RDL2004 Posts: 2,554
    edited 2013-03-13 09:03
    Hopefully IPv6 will come soon and save us from all that nonsense.

    I read somewhere recently that IPv6 would allow every grain of sand on the Earth to have its own ip address, and that there would be enough left over to provide the same to the beaches of another 340 similar planets. So hopefully this will not be an issue in the future. Maybe someone was actually thinking ahead for once :)
  • pedwardpedward Posts: 1,642
    edited 2013-03-13 12:52
    I have Comcast biz service with 1 static IP, cost is ~$70/mo and I run my own server, which is as fast as I want it to be. I've been doing the same thing 24/7 since 1999, but it was 1.5/384 DSL then.

    I don't mind the cost, because to get the same performance elsewhere would be costly and I wouldn't have any guarantee of performance. VPS providers oversubscribe anywhere from a little to a lot, and they don't seem to give a hoot. I had to migrate a couple of out-sourced servers in-house at work because performance was inconsistent and costly. Now they are assigned to dedicated cores on our in-house VPS servers.
  • softconsoftcon Posts: 217
    edited 2013-03-14 13:18
    If you need specific programs run, or if you need particular versions of software, then runing your own server is the only way to go. However, if you're just doing general hosting type things for web sites (I.E. providing web space, and providing programming for the web, pearl, php, scripts and so forth, then hosting your own server really isn't cost effective. I used to host web sites, as well as commercial gaming sites. The gaming sites paid well enough, but they sure took a toll on the servers, Round about 1999, or 2000, I stopped the gaming thing, and just kept the web hosting options. I'd been doing the hosting for 3-4 years at that point, and I'm sure mudconnect was happy to take my traffic, especially since until i pulled out of that market, I ranked higher in the search engings for game hosting than they did. :)
    I've discovered web hosting accounts hammer the servers less, eat much less resources (in geneeral, not always), and is truly cheaper to outsource to one of the mega corps that specialize in such things. My monthly cost is exactly nothing, unless I need to renew a domain, or re-up a hosting package, but even that spread out over 12 months isn't much more than 10 bucks a month. I can't get a decent internet connection for that price, much less pay the electric bill for running the servers required to host multiple domains. These days, my needs are few, so hosting offsite makes sense. If it ever becomes a problem, I'll certainly host my own again, but for now, the reseller thing works for me.
  • 4x5n4x5n Posts: 745
    edited 2013-03-14 22:38
    I've been a Unix systems Admin for the last 16 years and one of the few things I've learned over those years is that by itself the load average doesn't provide a good picture of the "health" of the server. All the load average tells you is how many processes are ready to run and waiting for a system resource. That could be CPU, disk, network, etc. I single user with seven processes dependent on an overloaded NFS server can jack the load up to 7+ without any performance issues for other processes. Same thing goes for an overloaded SAN connection.

    As a sys admin I've always looked at a load average over the number of cpus (now cores) as an indication that something should be looked into. It's not indicative of a problem though. I've seen servers regularly run at a load average of 4-5 per cpu without any performance issues. Among the worst offenders are active DB servers and file servers.

    I'm not saying that the server in question isn't overloaded but load average by itself means little. I have yet to find a single metric that will give you a clear indication of when a server is lightly, heavily or over loaded.
Sign In or Register to comment.