memory allocation in Linux

PROBLEM

I am running Apache HTTPD 1.3.37 with mod_php, when using the top

command, I see:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

12584 web 15 0 142m 12m 9176 S 8 0.3 2:07.01 httpd

12586 web 15 0 142m 12m 9168 S 8 0.3 2:08.85 httpd

12589 web 15 0 141m 10m 7376 S 8 0.3 2:06.83 httpd

12591 web 15 0 142m 11m 7420 S 8 0.3 1:59.64 httpd

12594 web 15 0 142m 11m 7444 S 8 0.3 2:06.25 httpd

12585 web 15 0 141m 12m 9200 S 6 0.3 2:03.83 httpd

Assume all the httpd process is show above...

QUESTION

1. Is the total memory currently used is : 12+12+10+11+11+12 = 68M ?

2. Is the max. memory even allocated by is: 142+142+141+142+142+141 = 850M

ANSWER

ANS1

justinp AT newmediagateway.com

Welcome to the wonderful world of Linux memory. The memory reported in the VIRT/RES column cannot simply be added up to calculate the total memory usage because some memory may be shared. When a process is fork()'d (to create a child process), the memory between the parent and child process will be shared. Once a portion of the memory is changed by the parent or child, that portion of the memory will become distinct between the two processes. For example:

* Parents using 10MB

* A child process is created

* Child now shows usage of 10MB, although total usage is only 10MB, not 20MB, due to shared memory

* Child process overwrites 5MB of the memory. Now that memory is unique to the child, so it will be separate

* Memory usage is now 10MB (parent) + 5MB (child) = 15MB (theoretically)

This is just a very basic example, since the true details behind the memory allocation is more complex. There is a python script available that will try to give you a more accurate calculation of memory usages available here:

http://www.pixelbeat.org/scripts/ps_mem.py

ANS 2

torsten.foertsch AT gmx.net

No and no.

How to interprete these figures depend a bit on your operating system.

But in general UNIX systems try to share memory between processes.

There are several ways how that can be achieved. In one way a program

module (executable binary, shared lib) is divided into sections. Each

section is marked by at compile time if it can be shared between

processes or not. Code sections are usually read-only during program

execution hence they can be shared. If your programs use a shared libc

for example all the code in this lib is shared between all processes

that use the lib. But a program module does not have to be loaded

completely into memory to do some work. Code or data pages that are not

needed are not loaded.

A second way to achieve memory sharing is called copy-on-write. When a

process forks both processes at first share all their memory. Now one

of them starts writing to a specific area of its memory. Only at this

time the operating system allocates a personal copy of that piece of

memory for that process. It also does not copy the whole process memory

but only a small chunk (page). So the more both processes write to

their memory the more memory they consume together. But they probably

never consume together twice the amount of memory one of them has

consumed while it has run alone.

So what you see as VIRT is the amount of mem that process would consume

if it runs completely alone and if all of its segments are loaded into

memory. RES is the part of VIRT that is actually allocated. Both say

nothing about sharing. SHR says how much of RES is shared between at

least 2 processes by the first method (not copy-on-write). But it does

not say among how much or which processes it is shared.

Hence you can't simply add up these numbers.

A better way to judge memory consumption on Linux is /proc/$PID/smaps

and tools that use it. /proc/$PID/clear_refs on recent kernels is also

interesting in that regard. To get a general impression how much memory

my apache needs you can start vmstat with a small parameter, stop the

apache and look how it affects the memory related columns.

--