While porting .NET (NativeAOT) to KasperskyOS CE 1.4.0.102, I collected where the SDK behaves differently from POSIX or Linux, or lacks something portable code expects. Most findings are shown by a small program run as the only application in a QEMU image, and most of those also on Linux for comparison; the others by commands run on the installed SDK. FINDINGS.md in https://github.com/caraioniurie47/kos-sdk-checks describes each one (the output, what POSIX, NetBSD and Linux do, what it breaks, and for most of them options), and the repository has the programs and scripts that reproduce them; the VfsRamFs crash has a reproducer of its own.

Bugs

• B1 — mkstemps() fails with EINVAL for a valid template

• B2 — pread, pwrite, ftruncate, fsync fail with ENOSYS on /dev/null, and three of them on pipes

• B3 — VfsRamFs: utimensat(), rename() and posix_fallocate() fail unlike POSIX

• B4 — VfsRamFs crashes under concurrent stat/readdir/unlink (reported in https://forum.kaspersky.com/topic/vfsramfs-crashes-null-dereference-in-inode_destructor-under-concurrent-statreaddirunlink-kasperskyos-ce-140102-59792/)

• B5 — shutdown() on an unconnected TCP or UDP socket succeeds

• B6 — a socket from accept4() inherits the listener's O_NONBLOCK

• B7 — poll() fails with EBADF for a closed descriptor instead of setting POLLNVAL

• B8 — a connected UDP socket cannot be disconnected (connect(AF_UNSPEC))

• B9 — a zero-length recv() on an empty non-blocking TCP socket returns 0

• B10 — sendto() of zero bytes on UDP returns 0 and sends nothing

• B11 — IOV_MAX is 10, and more iovecs give EINVAL, not EMSGSIZE

• B12 — a read() or recvmsg() blocked on a socket closed by another thread fails with errno -3

• B13 — at the 512-descriptor limit open("/dev/null") fails with ENFILE; getrlimit/setrlimit are not implemented

• B14 — sigaction() installs a handler for SIGKILL and SIGSTOP

• B15 — mprotect() to read, write and execute fails with ENOMEM, not ENOTSUP

• B16 — dlerror() is declared const char *

Documentation errors

• E1 — link() fails with ENOSYS, though the manual lists it

• E2 — sendfile() to a TCP socket fails with EINVAL, though the manual lists it

• E3 — the manual documents IPv6 for kos_net.h, but the network stack has no IPv6

• E4 — two typos in log messages

Undocumented behaviour

• U1 — uid 0 without root's rights, yet unlink() removes directories; directories lose setuid and setgid

• U2 — VfsRamFs's statvfs() reports no free space

• U3 — readdir() of a removed directory fails with ENOENT

• U4 — localhost resolves only from a hosts file given to VfsNet

• U5 — FIONREAD on UDP counts 16 bytes more than the datagram

• U6 — SO_SNDBUF/SO_RCVBUF of 0 fail with EINVAL

• U7 — getsockopt() with a NULL buffer and length 0 fails with EINVAL

• U8 — TCP sendmsg() of more than 64 KiB fails with EMSGSIZE, even on a blocking socket

• U9 — accept4() with address_len over 128 fails with EACCES

• U10 — setsockopt() fails with ECONNRESET after the peer's reset

• U11 — accept() with no descriptor left drops the pending connection

• U12 — uname() returns constants

• U13 — pthread_condattr_init() fails on memory that holds an initialized attribute

• U14 — the VFS client waits about 10 s at startup when there is no VFS server

• U15 — PROT_NONE reservations take physical memory and seconds; MADV_DONTNEED frees nothing

• U16 — when a MAP_NORESERVE write finds no memory, the kernel ends the process

• U17 — mprotect() cannot make a read-only page of the program image writable

• U18 — aarch64-kos-clang -static-pie links dynamically

• U19 — unprefixed clang and clang-17 in toolchain/bin target KasperskyOS

Missing features

• M1 — SO_REUSEPORT works but no header defines it

• M2 — sysconf(_SC_PHYS_PAGES) fails although unistd.h defines the name

• M3 — OpenSSL 1.1.1t libraries without headers; OpenSSL 3

• M4 — libcrypto.a references a getentropy() that nothing provides

• M5 — no GSSAPI or Kerberos library

• M6 — functions declared in headers but defined in no library

• M7 — no SA_RESETHAND or SA_NODEFER

• M8 — no C <uchar.h>

• M9 — fallocate() without its FALLOC_FL_* flags

• M10 — commonly probed headers are absent

Proposals

• P1 — an API to suspend another thread and read its registers

• P2 — a supported way for an exception handler to resume at a new address with new registers

• P3 — a way to query the CPU's features

Which of these do you plan to fix, which to document, and should any of them get a topic of its own?

Context

Found while porting .NET (NativeAOT) to KasperskyOS (latest release: https://github.com/caraioniurie47/runtime-kos/releases/latest).

Thanks!

Very interesting findings. Hope support will create related internal tickets.

21 hours ago, Yura Beznos said:

Very interesting findings. Hope support will create related internal tickets.

Thanks! I'm waiting for feedback too. Before I publish the next runtime-kos release, I'd like to know which of these items the SDK team plans to fix, which ones they'll document as intended behaviour, and which ones they'll leave as they are. The answer decides whether each workaround in the port stays in, gets a pointer to the documentation, or can be dropped. Even a short answer on a few items would help.

@HQCS hey, hello. It looks like those findings might be very interesting for KOS CE team. What do you think?

Hello!

An incident INC000018676317 has been created. We will return to you once we have any news.