Odd Directory Server problem

So, I’ve run into a bit of an odd problem with Sun’s LDAP Directory Server (the latest version - 5.2P3). Occasionally, Linux clients have their authentications hang. Everything will be fine, and then you’ll request another shell (or need a process created or whatever) and on the surface at least it seems that the connection the client opens to the LDAP server just stalls. Often the first time you log into a box the auth will never succeed, but any subsequent attempts will fail.

Now, fortunately whenever this happens I get what you’d think would be a fairly useful error message on the server:

Jun 14 09:55:04 ldap01 ns-slapd[21169]: [ID 555009 user.alert] crypt: fdopen(266) failed: Too many open files

The curious part is that AFAIK the ns-slapd process has setrlimit()ed its file descriptor max up to 65K:

bash-3.00# plimit /proc/21169
21169:  ./ns-slapd -D /var/opt/mps/serverroot/slapd-ldap01 -i /var/opt/mps/ser
resource              current         maximum
time(seconds)         unlimited       unlimited
file(blocks)          unlimited       unlimited
data(kbytes)          unlimited       unlimited
stack(kbytes)         8480            130336
coredump(blocks)      unlimited       unlimited
nofiles(descriptors)  65536           65536
vmemory(kbytes)       unlimited       unlimited

The Directory Server Console app also reports that it has 65K connections available, yet when I start seeing the problem the tool reports “Open Connections” in the 200+ range, but never more than 256. It’s almost like the file descriptor soft limit (which defaults to 256) is in fact being enforced somehow. A normal shell does report a limit of 256:

bash-3.00# ulimit -a
core file size        (blocks, -c) unlimited
data seg size         (kbytes, -d) unlimited
file size             (blocks, -f) unlimited
open files                    (-n) 256
pipe size          (512 bytes, -p) 10
stack size            (kbytes, -s) 8480
cpu time             (seconds, -t) unlimited
max user processes            (-u) 8357
virtual memory        (kbytes, -v) unlimited

So. Who’s telling the truth? Everything points to the process hitting the 256 open file soft limit, yet plimit and the tool itself claim they’ve got all 65K descriptors available.

While this problem started with a “stock” Directory Server 5.2 running Solaris 9, I’ve since moved to the latest Patch 3 release of Directory Server running on Solaris 10 just to see if perhaps there was some weird bug in Solaris 9 x86. But it’s the exact same behaviour on both setups.

I suppose the next step is to set rlim_fd_cur to something like 2048, reboot, and see if the problem magically goes away. I just hate it when I get an apparently inconsistent view of the system, though. So perhaps before I do the /etc/system update I’ll do some DTrace reading and see if there’s something there that might enlighten me.

Of course, the other interesting thing to ponder is why these Linux pam_ldap clients seem to open so many freakin’ connections to the LDAP server. For example, just from the mail server:

bash-3.00# netstat -a|grep ESTABLISHED|grep mercury|wc -l
59

60 connections. A relatively idle shell server has 24 connections open right now. Seems like a lot. Zope and its LDAPUserFolder, by contrast, only opens at most 3 connections yet it probably does as many or more auths per second as the Linux shell server. Is this normal?

Update: So on a hunch that ns-slapd was somehow reverting back to a 256 open file limit after it switches uid to the ldap user, I decided to update /etc/system and “set rlim_fd_cur=65536“. After reboot now even the unpriveleged ldap account should be able to open 65K files:

-bash-3.00$ id
uid=389(ldap) gid=389(ldap)
-bash-3.00$ ulimit -n
65536

Yet I still see the same problem, and again I *never* see more than 230 or so open connections by the ns-slapd process even when it’s spewing the “too many open files” errors. Frustrating problem, to say the least. The bit that has me curious now is what role “crypt” has to play in all of this. Most of my passwords are indeed in crypt format. Perhaps the open file limit is being hit in making calls to crypt() on password checks?

Update 2: Well just out of curiosity I wanted to see what ns-slapd was up to. So I ran a “prstat -m -p 525” on it and saw that most of the time was spent on systems calls and voluntary context switches (the two go hand in hand of course). Curious what the syscall load was I found this little DTrace script in the DTrace Guide:

root@ldap01:/var/opt/mps# dtrace -n syscall:::entry'/pid == 525/{ @syscalls[probefunc] = count(); }'
dtrace: description 'syscall:::entry' matched 229 probes
^C

lwp_exit                                                          1
fstat64                                                           1
lwp_sigmask                                                       1
schedctl                                                          1
setcontext                                                        1
lwp_continue                                                      1
lwp_create                                                        1
lwp_kill                                                          1
close                                                             3
setsockopt                                                        3
accept                                                            3
getsockname                                                       3
statvfs64                                                        33
read                                                            100
gtime                                                           134
lwp_park                                                        192
write                                                           474
pollsys                                                         615

So it’s really nothing surprising. The main thread is poll()ing on the socket and of course writing some data back over the network to clients. So no big break yet. Tomorrow I’m going to see if I can convert some of the passwords to native MD5 (vs. crypted MD5s) and do a few specific connection tests with a LDAP load script I wrote up in Ruby today.

Update 3: So my hunch that crypt was somehow the culprit was right. The reason, however, wasn’t what I expected. And determing the problem without the existence of OpenSolaris probably wouldn’t have been possible. After reading this warning from “man fdopen” I knew it was something in the stdio library that was getting me:

The minimum number of streams that a process can have open at one time is FOPEN_MAX. For 32-bit applications, however, the underlying ABIs require that no file descriptor used to access the file underlying an stdio STREAM have a value greater than 255.

fdopen() uses a stdio STREAM, so that’s where I’m hitting the wall. Now, why would crypt be using fdopen() from stdio at all? I floated the question in #opensolaris and happily dprice and sommerfeld offered insight. First of all, these guys are very approachable, and open about the limits of the AT&T stdio implementation that Solaris uses. After a little grumbling/joking talk about “BSD got it right the first time”, and “Thank you for using.. AT&T”, I found out that as I expected, the 255 fd limit on the Solaris 32-bit ABI has a “long, and very sad saga behind it”. That’s not surprising, of course. What was surprising was just talking to kernel engineers about the issue, how willing they were to help, and how they openly discussed how they might work around it. So refreshing. So… fantastic, really. Back to why ns-slapd was running into fdopen() limits. Dan looked at crypt.c and had a hunch that it was trying to read /etc/security/crypt.conf and that was the problem. He dug up a Dtrace script for me to get a stack trace when the fdopen() call returned null. It didn’t take long to get a result:

root@ldap01:/etc/security#  dtrace -p 525 -n 'pid$target::fdopen:return/arg1 == 0/{ustack();}'
dtrace: description 'pid$target::fdopen:return' matched 1 probe
Jun 16 16:04:59 ldap01 ns-slapd[525]: [ID 555009 user.alert] crypt: fdopen(294) failed: Too many open files
CPU     ID                    FUNCTION:NAME
0  41216                    fdopen:return
libc.so.1`0xcd78422a
libc.so.1`crypt+0x88
pwdstorage-plugin.so`crypt_pw_cmp+0x2b
libslapd.so.1`slapi_pw_find_sv+0x6a
libback-ldbm.so`ldbm_back_bind+0x140
libfe.so`bind_core_pb+0x56d
libfe.so`dispatch_operation_core_pb+0xdb
libfe.so`0xce6a024b
libfe.so`ldap_frontend_main_using_core_api+0xa7
libfe.so`generic_handle_read_data+0x95
libfe.so`0xce6a1244
libnspr4.so`_pt_root+0xc4
ns-slapd`0x81078e0
0x4d580000

The top call is to getalgbyname(), which reads crypt.conf to figure out which .so library to load for the algorithm in question. So there was the culprit! Phew! So now I have bug 6287034 open, but of course a kernel workaround isn’t going to magically appear tomorrow which puts me back at square one. I.e. how do I get this LDAP server fixed and into production?

Right now the plan is pretty simple: I’ll bring a shell server over to pam_ldap but leave SSL off so I can sniff out the LDAP packets and grab user’s plain text passwords. A little python or ruby script can do the filtering for me. Once I have the passwords I’ll put them into the directory server as “native” SSHA hashes. That should eliminate the crypt_unix() calls that ns-slapd makes to check a user’s password.

About this entry

You’re currently reading “Odd Directory Server problem,” an entry on VMUNIX Blues

Published:: 06.14.05 / 10am

Category:: general, solaris, ldap

nathan hruby 06.15.05 / 7am

Is nscd running on the linux hosts? Probably it isn’t and the LDAP conenctions are for nss_ldap doing various nss lookups over and over again (assuming of course, you have nss_ldap turned on, which I’m blindly, and possibly wrongly, assuming you do since you have pam_ldap on..)

mark 06.15.05 / 11am

I turned nscd off after I was seeing some weirdness with it blocking some user accounts in LDAP. Weird stuff where if you do a “getent passwd username” with nscd off the record shows up just fine. Turn nscd on and you get nothing back. You’re correct of course that with it on I see a lot less nss traffic to the LDAP server. But I still see the problem.

Jason 06.16.05 / 9pm

Reading the tunables guide, I came across this:

Compared to rlim_fd_max. If rlim_fd_cur is greater than rlim_fd_max, rlim_fd_cur is reset to rlim_fd_max.

You might need to also increase rlim_fd_max. Having said that, rlim_fd_max section mentions this:

A 32-bit program using standard I/O is limited to 256 file descriptors. A 64-bit program using standard I/O can use up to 2 billion descriptors. Specifically, standard I/O refers to the stdio(3C) functions in libc(3LIB).

Not sure if this is the crux of the problem but it might be since this program is running on a 32 bit system.

Jim Litchfield 06.20.05 / 7am

The key is whether the LDAP server is a 32-bit app. It probably is and there is a limitation
of 256 FDs that can be open in the stdio library for 32-bit apps. Given the error message,
this is most likely the cause. You could harass the support people to ask the LDAP server
people to make sure that they don’t have any fds in the range of 0-255 that don’t use
stdio. Although that would make life easier, the real cure is to make the LDAP server 64-bit.

There were several efforts to “enhance” the 32-bit library to support more than 256
fds but these all foundered on various obscure edge cases that meant one would have to
recompile all apps and libraries that used 32-bit to be totally safe. The sticking point?
Mainly fileno() which for a very long time was a macro referencing an unsigned byte field.

Steve Lee 07.15.05 / 10am

I am currently running into the same problem you have. What was your final solutions to this.

mark 07.15.05 / 11am

Steve: Don’t use {crypt} style password entries in your directory, basically. If you load the passwords in as {SSHA} hashes in your directory, ns-slapd doesn’t have to call out to the UNIX crypt() routines in libc, so you don’t hit the stdio file descriptor limit.

If you only have UNIX crypts from, say, /etc/passwd, you have to get users to re-enter their passwords. We made a simple script to do this. Painful, yes. But that’s the only solution if you want to stick with Directory Server on a 32-bit system.

The other solution would be to use OpenLDAP. Or get a 64-bit Solaris box…

David Baldwin 08.03.05 / 4pm

OpenLDAP uses crypt() in the same circumstances and with the same limitations on my Solaris 9 box.

from man -s 3c crypt:
The crypt() function calls crypt_gensalt(3C) to generate the
salt. If the first character of salt is “$”, crypt() uses
crypt.conf(4) to determine which shared module to load for
the encryption algorithm. If the first character of salt is
not “$”, the algorithm described on crypt_unix(5) is used.

And it can’t open /etc/security/crypt.conf if it doesn’t have a free file descriptor available

mark 08.04.05 / 2pm

Thanks for the clarification, David. I should have said “use OpenLDAP on a non-Solaris box” as a possible solution. FreeBSD and Linux, for example, don’t have the stdio limitation so you wouldn’t bump into the problem.

We chose to stick with Sun Directory Server on Solaris 10 (i386) and just got people to re-enter their passwords. But if I were starting from scratch again I’d probably just go OpenLDAP on Linux.

OpenDS - YALDAPS and a look at the current playing field at VMUNIX Blues 08.02.06 / 5pm

[…] If I had to summarize the differences between Netscape DS and OpenLDAP, it would be this: One is well documented, and one isn’t. In case you didn’t guess, the Netscape LDAP server is the one that’s leaps and bounds better in this regard and now that Fedora DS is out there for free, it’s the one I recommend any sysadmins starting with LDAP take a look at first. Even if you don’t deploy Fedora DS, or Red hat DS, or Sun DS (on 64-bit Solaris — stay far, far away from Sun DS on Linux or 32-bit Solaris) you need to start with the Netscape docs. Because only they actually have chapters about general concepts, systems architecture, and deployment scenarios. The OpenLDAP docs will tell you how to use OpenLDAP, but they won’t tell you how to build directory services. Which means you’ll probably give up and walk away from OpenLDAP before you make it do anything useful for you. […]