open files

[Roshan]

Hello Erman,

We currently have openfiles set for our application as 'test hard  nofile  65536'. But application crash because of error 'too many open files'. To what maximum limits can I set it?

Regards,

Roshan
'

[ErmanArslansOracleBlog]

Check the apps processes and  see if they are getting the correct/updated limits..
Check it using /proc filesystem while the applications are running.. Get one the application pids and get its limits from /proc.

Ensure you have modified the limits.conf correctly. (that "test"  over there is the OS user name bytheway.. so your OS user name is test ?)

Ensure you don't have any other script that overrides your setting in limits.conf.. connect with the OS user that owns the applications and check with ulimit command.

For instance -> in Redhat 5,6,7,8 there is a note like below;

Note that the nproc setting can no longer be set in limits.conf. Please use /etc/security/limits.d/90-nproc.conf instead. Setting nproc in /etc/security/limits.conf has no effect in Red Hat Enterprise Linux.

There is an option for unlimited setting as well.. see https://linux.die.net/man/5/limits.conf

So you write unlimited rather than the value.. However; making the limits unlimited may be a dangerous move in some environments.. I mean, misbehaving apps/process can get very high resources as there will be no limit for stopping them.

[ErmanArslansOracleBlog]

Also ensure that you have the proper setting for  fs.file-max  in sysctl.conf. It is a system wide limit for files.. The ulimits are per user.. So you need to have both setting properly in place..
Note that if you modify the sysctl.conf, you need to execute sysctl -p command to reflect the changes.

Also note that, if this is due to an NFS share, then ulimit should be increased on the NFS server side, rather than the client side/Linux client side.. Just saying..

[Roshan]

Hi Erman,

yes you are write. In fact Striim is reading golden gate trail files from an NFS share. Also, it is reading flat files from another NFS share. I will increase the ulimit on both NFS server sides.

Thanks for your help.

[Roshan]

Suppose I run ps -ef | grep striim
the pid is 12886

I do  an ls -ltr 12886 from /proc

the below files and directories are listed

lrwxrwxrwx.    1 striim striim 0 May 28 22:27 exe -> /usr/java/jdk1.8.0_271-amd64/bin/java
-r--r--r--.    1 striim striim 0 May 28 22:27 mountinfo
-r--r--r--.    1 striim striim 0 May 28 22:27 maps
-r--r--r--.    1 striim striim 0 May 28 22:27 cgroup
-r--r--r--.    1 striim striim 0 May 28 22:27 status
dr-xr-xr-x.    7 striim striim 0 May 28 22:27 net
-r--r--r--.    1 striim striim 0 May 28 22:28 stat
-r--r--r--.    1 striim striim 0 May 28 22:28 mounts
dr-x------.    2 striim striim 0 May 28 22:28 fd
-r--r--r--.    1 striim striim 0 May 28 22:28 statm
-r--r--r--.    1 striim striim 0 May 28 22:28 cmdline
dr-xr-xr-x. 1251 striim striim 0 May 28 22:37 task
-r--r--r--.    1 striim striim 0 May 29 15:36 wchan
-rw-r--r--.    1 striim striim 0 May 29 15:36 uid_map
-r--r--r--.    1 striim striim 0 May 29 15:36 timers
-r--r--r--.    1 striim striim 0 May 29 15:36 syscall
-r--r--r--.    1 striim striim 0 May 29 15:36 stack
-r--r--r--.    1 striim striim 0 May 29 15:36 smaps
-rw-r--r--.    1 striim striim 0 May 29 15:36 setgroups
-r--r--r--.    1 striim striim 0 May 29 15:36 sessionid
-r--r--r--.    1 striim striim 0 May 29 15:36 schedstat
-rw-r--r--.    1 striim striim 0 May 29 15:36 sched
lrwxrwxrwx.    1 striim striim 0 May 29 15:36 root -> /
-rw-r--r--.    1 striim striim 0 May 29 15:36 projid_map
-r--r--r--.    1 striim striim 0 May 29 15:36 personality
-r--------.    1 striim striim 0 May 29 15:36 patch_state
-r--r--r--.    1 striim striim 0 May 29 15:36 pagemap
-rw-r--r--.    1 striim striim 0 May 29 15:36 oom_score_adj
-r--r--r--.    1 striim striim 0 May 29 15:36 oom_score
-rw-r--r--.    1 striim striim 0 May 29 15:36 oom_adj
-r--r--r--.    1 striim striim 0 May 29 15:36 numa_maps
dr-x--x--x.    2 striim striim 0 May 29 15:36 ns
-r--------.    1 striim striim 0 May 29 15:36 mountstats
-rw-------.    1 striim striim 0 May 29 15:36 mem
dr-x------.    2 striim striim 0 May 29 15:36 map_files
-rw-r--r--.    1 striim striim 0 May 29 15:36 loginuid
-r--r--r--.    1 striim striim 0 May 29 15:36 limits
-r--------.    1 striim striim 0 May 29 15:36 io
-rw-r--r--.    1 striim striim 0 May 29 15:36 gid_map
dr-x------.    2 striim striim 0 May 29 15:36 fdinfo
-r--------.    1 striim striim 0 May 29 15:36 environ
lrwxrwxrwx.    1 striim striim 0 May 29 15:36 cwd -> /opt/striim
-r--r--r--.    1 striim striim 0 May 29 15:36 cpuset
-rw-r--r--.    1 striim striim 0 May 29 15:36 coredump_filter
-rw-r--r--.    1 striim striim 0 May 29 15:36 comm
--w-------.    1 striim striim 0 May 29 15:36 clear_refs
-r--------.    1 striim striim 0 May 29 15:36 auxv
-rw-r--r--.    1 striim striim 0 May 29 15:36 autogroup
dr-xr-xr-x.    2 striim striim 0 May 29 15:36 attr
-bash-4.2$

I chcked the limits file. I see the max open file is only 4096!

-bash-4.2$ cat limits
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        0                    unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             1031064              1031064              processes
Max open files            4096                 4096                 files
Max locked memory         65536                65536                bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       1031064              1031064              signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us

Should I increase it?

[ErmanArslansOracleBlog]

If you check the limits of an app process using /proc filesystem and if you see 4096 there, it means either it is not getting the update limits or it needs a restart..

Check my previous update to make sure you are setting the limits properly.. These lines especially ->

Ensure you don't have any other script that overrides your setting in limits.conf.. connect with the OS user that owns the applications and check with ulimit command.

For instance -> in Redhat 5,6,7,8 there is a note like below;

Note that the nproc setting can no longer be set in limits.conf. Please use /etc/security/limits.d/90-nproc.conf instead. Setting nproc in /etc/security/limits.conf has no effect in Red Hat Enterprise Linux.