pam_strerror()required the PAM handle to be passed to it, while the RFC specifies the opposite. In the other case, the response structure is filled differently. A response item is returned for each message sent to the conversation function (if I recall correctly) instead of only for "prompt" messages sent. So modules written to work with Linux PAM versions 0.58 and before will need to be updated to work with either newer versions of Linux PAM or with Solaris PAM.
setcred()API was being used (e.g. ESTABLISH, REFRESH, DESTROY). In Linux PAM 0.62 patches I contributed to fix this appeared; When compiled on a Solaris machine Linux PAM uses flags which match those used by Solaris instead of by Linux. This means that modules compiled under Solaris will never work in binary compatibility under SparcLinux, but little else.
pam_sm_setcred()portion *after* the setuid happens. The Solaris login calls it before. This means you potentially need to chown things like Kerberos ticket files yourself. Be careful, though, that you're not chowning a symlink to something else in the process, if you need to do something like this in your code!
pam_set_item(PAM_USER); It expects this data to be filled in by
pam_get_user()only, and not for the module to explicitly try to set it. This is fine, it just means you need to make your module work in this manner.
pam_putenv(). This is currently in SunSolve as bug 4091713. These appear to be fixed by patch 105703-03
login auth required /usr/lib/security/pam_NEW_auth.so.1 telnet auth requisite /usr/lib/security/pam_NEW_auth.so.1 telnet auth required /usr/lib/security/pam_unix.so.1