Difference between revisions of "Resource Limits on the Linux Clients"
| (6 intermediate revisions by one other user not shown) | |||
| Line 5: | Line 5: | ||
== Memory Usage == | == Memory Usage == | ||
| − | The maximum memory (RAM) usage for a program is ''' | + | The maximum memory (RAM) usage for a program is '''16 GiB'''. This limit applies only to the non-lab systems like ugradx and ugradz. The lab systems (ugrad1 through ugrad24) have no imposed limit, so it's possible for a buggy program to use up all of a system's RAM. If one lab computer is unresponsive, you can simply use a different one until we're able to fix the problem. (But, email [mailto:support@cs.jhu.edu support@cs.jhu.edu] to let us know you experienced an issue, and include the system name as well.) |
| − | Note that this limit applies collectively to every program started from the ''same'' login session. Running one program that uses | + | Note that this limit applies collectively to every program started from the ''same'' login session. Running one program that uses 16 GiB of RAM will hit the limit, but so will running eight programs that each use 2 GiB of RAM. |
| + | |||
| + | The limit also applies to memory reservations, whether or not they're actively being used. This means if you <code>mmap</code> a file, the entire file size will count towards this limit, even though blocks from the file are only paged into RAM on demand. | ||
When your program hits the memory limit, any further attempts to allocate memory will fail. How that failure is manifested will depend on your programming language. In C, <code>malloc()</code> (or another memory allocation function) will return <code>NULL</code> and <code>errno</code> will be set to <code>ENOMEM</code>. In Python, a <code>MemoryError</code> exception will be raised. In Java, an <code>OutOfMemoryError</code> will be raised. For other languages, please consult the relevant documentation. | When your program hits the memory limit, any further attempts to allocate memory will fail. How that failure is manifested will depend on your programming language. In C, <code>malloc()</code> (or another memory allocation function) will return <code>NULL</code> and <code>errno</code> will be set to <code>ENOMEM</code>. In Python, a <code>MemoryError</code> exception will be raised. In Java, an <code>OutOfMemoryError</code> will be raised. For other languages, please consult the relevant documentation. | ||
| Line 13: | Line 15: | ||
== Number of Processes == | == Number of Processes == | ||
| − | We have a limit of ''' | + | We have a limit of '''8192 process threads''' per account on each of our Linux clients. |
| − | Note that this limit applies to '' | + | Note that this limit applies to ''threads'', not just processes. Every process has at least one thread, but many processes run using more than one thread. You can see a list of all of your processes from the command line by running <kbd>ps x</kbd> . You can see a list of all your process threads by running <kbd>ps x -L</kbd> . |
| − | If the total number of running processes started from the same | + | If the total number of running processes started from the same account is at the limit, attempts to start any additional processes will fail. How that failure is manifested will depend on the program that tried to launch the additional process. In C, <code>fork()</code> will return -1 and <code>errno</code> will be set to <code>EAGAIN</code>. A shell will typically give a message along the lines of "fork failed: Resource temporarily unavailable". In Python, you will typically get a <code>BlockingIOError</code> with a message of "[Errno 11] Resource temporarily unavailable". The Java exception raised will vary depending on the function having been called; for instance, <code>java.util.concurrent.ForkJoinPool.execute()</code> will raise <code>java.util.concurrent.RejectedExecutionException</code>. |
== Maximum Process Runtime == | == Maximum Process Runtime == | ||
Latest revision as of 15:07, 8 November 2024
Our undergraduate Linux clients have system-enforced limits on some programs' resource usage. These limits are designed to prevent runaway programs from impacting other people's use of the shared computers. The limits should not affect your routine use of the systems; if you believe they're causing problems, please contact us to discuss the issue.
For some guidance on not exceeding these limits, please see our page on Preventing Your Programs From Overrunning Our Computers.
Memory Usage
The maximum memory (RAM) usage for a program is 16 GiB. This limit applies only to the non-lab systems like ugradx and ugradz. The lab systems (ugrad1 through ugrad24) have no imposed limit, so it's possible for a buggy program to use up all of a system's RAM. If one lab computer is unresponsive, you can simply use a different one until we're able to fix the problem. (But, email support@cs.jhu.edu to let us know you experienced an issue, and include the system name as well.)
Note that this limit applies collectively to every program started from the same login session. Running one program that uses 16 GiB of RAM will hit the limit, but so will running eight programs that each use 2 GiB of RAM.
The limit also applies to memory reservations, whether or not they're actively being used. This means if you mmap a file, the entire file size will count towards this limit, even though blocks from the file are only paged into RAM on demand.
When your program hits the memory limit, any further attempts to allocate memory will fail. How that failure is manifested will depend on your programming language. In C, malloc() (or another memory allocation function) will return NULL and errno will be set to ENOMEM. In Python, a MemoryError exception will be raised. In Java, an OutOfMemoryError will be raised. For other languages, please consult the relevant documentation.
Number of Processes
We have a limit of 8192 process threads per account on each of our Linux clients.
Note that this limit applies to threads, not just processes. Every process has at least one thread, but many processes run using more than one thread. You can see a list of all of your processes from the command line by running ps x . You can see a list of all your process threads by running ps x -L .
If the total number of running processes started from the same account is at the limit, attempts to start any additional processes will fail. How that failure is manifested will depend on the program that tried to launch the additional process. In C, fork() will return -1 and errno will be set to EAGAIN. A shell will typically give a message along the lines of "fork failed: Resource temporarily unavailable". In Python, you will typically get a BlockingIOError with a message of "[Errno 11] Resource temporarily unavailable". The Java exception raised will vary depending on the function having been called; for instance, java.util.concurrent.ForkJoinPool.execute() will raise java.util.concurrent.RejectedExecutionException.
Maximum Process Runtime
Any program that has consumed more than 1 week of accumulated CPU time will be terminated.
"Accumulated CPU time" is the total amount of time spent actively using a single CPU core. For example, a single-threaded process running continuously for one week will accumulate one week of CPU time. Along the same lines, a process with seven threads, all running continuously, will accumulate seven seconds of CPU time for every actual elapsed second and, therefore, will accumulate one week of CPU time after running for one day.
When a program exceeds the CPU time limit, it will be sent the SIGXCPU signal. If the program has no registered handler for that signal, it will simply stop running and its parent process will be notified. If it has a registered signal handler, the handler will be given about 30 seconds of runtime to clean up any resources before the process is ended. The process will be sent a new SIGXCPU signal every second until it exits. If it's still running after 30 seconds, the process will be forcibly terminated.
Disk Usage
Ugrad systems are also subject to disk usage limits, as described on the Disk Quotas page.
If your disk usage is at its limit, attempts to write additional data to files in the /home filesystem will fail. In C, the write() function will return -1 and errno will be set to EDQUOT (or, in some cases, ENOSPC). Python will raise an OSError exception with message "[Errno 122] Disk quota exceeded". Java might not raise an exception and instead might just silently fail to write the data.