ClamAV 0.103.0

What's new in this version:

With your feedback on the previous candidates, we've fixed these additional issues:
- The freshclam PID file was not readable by other users in previous release candidates but is now readable by all.
- An issue with how freshclam was linked with the autotools build system caused SysLog settings to be ignored.
- The real-path checks introduced to clamscan and clamdscan in 0.102.4 broke scanning of some files with Unicode filenames and files on network shares for Windows users.
- Thanks to the users for your help in fixing these bugs.

Major changes:
- clamd can now reload the signature database without blocking scanning. This multi-threaded database reload improvement was made possible thanks to a community effort.
- Non-blocking database reloads are now the default behavior. Some systems that are more constrained on RAM may need to disable non-blocking reloads, as it will temporarily consume double the amount of memory. We added a new clamd config option ConcurrentDatabaseReload, which may be set to no.

Notable changes:
- The DLP module has been enhanced with additional credit card ranges and a new engine option that allows ClamAV to alert only on credit cards (and not, for instance, gift cards) when scanning with the DLP module. John Schember developed this feature, with input from Alexander Sulfrian.
- We added support for Adobe Reader X PDF encryption and overhauled the PNG-scanning tool to detect PNG-specific exploits. We also made a major change to GIF parsing that now makes it more tolerant of problematic files and adds the ability to scan overlays, all thanks to work and patches submitted by Aldo Mazzeo.
- clamdtop.exe is now available for Windows users. The functionality is somewhat limited when compared to clamdtop on Linux. PDCurses is required to build clamdtop.exe for ClamAV on Windows.
- The phishing detection module will now print "Suspicious link found!" along with the "Real URL" and "Display URL" each time ClamAV detects phishing. In a future version, we would like to print out alert-related metadata like this at the end of a scan, but for now, this detail will help users understand why a given file is being flagged as phishing.
- Added new *experimental* CMake build tooling. CMake is not yet recommended for production builds. Our team would appreciate any assistance improving the CMake build tooling so we can one day deprecate autotools and remove the Visual Studio solutions.
- Please see the new CMake installation instructions found in INSTALL.cmake.md for detailed instructions on how to build ClamAV with CMake.
- Added --ping and --wait options to the clamdscan and clamonacc client applications.
- The --ping (-p) command will attempt to ping clamd up to a specified maximum number of attempts at an optional interval. If the interval isn't specified, a default one-second interval is used. It will exit with status code `0` when it receives a PONG from clamd or status code `21` if the timeout expires before it receives a response.
- Added Excel 4.0 (XLM) macro detection and extraction support. Significantly improved VBA detection and extraction as well. Work courtesy of Jonas Zaddach.
- This support not yet added to sigtool, as the VBA extraction feature in sigtool is separate from the one used for scanning and will still need to be updated or replaced in the future.
- Improvements to the layout and legibility of temp files created during a scan. Improvements to legibility and content of the metadata JSON generated during a scan.
- To review the scan temp files and metadata JSON, run:
- clamscan --tempdir=<path> --leave-temps --gen-json <target>
- Viewing the scan temp files and metadata.json file provides some insight into how ClamAV analyzes a given file and can also be useful to analysts for the initial triage of potentially malicious files.

Other improvements:
- Added ability for freshclam and clamsubmit to override default use of OpenSSL CA bundle with a custom CA bundle. On Linux/Unix platforms (excluding macOS), users may specify a custom CA bundle by setting the CURL_CA_BUNDLE environment variable. On macOS and Windows, users are expected to add CA certificates to their respective system's keychain/certificate store. Patch courtesy of Sebastian A. Siewior.
- clamscan and clamdscan now print the scan start and end dates in the scan summary.
- The clamonacc on-access scanning daemon for Linux now installs to sbin instead of bin.
- Improvements to the freshclam progress bar to ensure the text does not shift around as information changes. The bar will not exceed 80 characters, even on very slow connections. Time is now displayed in Xm XXs (or Xh XXm) for values of 60 seconds or longer. The bytes display now changes units at the proper 1024 B/KiB instead of 2048 B/KiB. Patch courtesy of Zachary Murden.
- Improve column alignment and line wrap rendering for clamdtop. Also fixed an issue on Windows where clamdtop would occasionally disconnect from clamd and fail to reconnect. Patch courtesy of Zachary Murden.
- Improvements to the AutoIT parser.
- Loosened the curl version requirements in order to build and use clamonacc. You may now build ClamAV with any version of libcurl. However, clamonacc's file descriptor-passing (FD-passing) capability will only be available with libcurl 7.40 or newer. FD-passing is ordinarily the default way to perform scans with clamonacc, as it is significantly faster than streaming.
- Added LZMA and BZip2 decompression routines to the bytecode signature API.
- Disabled embedded type recognition for specific archive and disk image file types. This change reduces file type misclassification and improves scan time performance by reducing duplicated file scanning.
- Use pkg-config to detect libpcre2-8 before resorting to pcre2-config or pcre-config.
- Patch courtesy of Michael Orlitzky.

Bug fixes:
- Fixed issue scanning directories on Windows with clamdscan.exe that was introduced when mitigating against symlink quarantine attacks.
- Fixed behavior of freshclam --quiet option. Patch courtesy of Reio Remma.
- Fixed behavior of freshclam's OnUpdateExecute, OnErrorExecute, and OnOutdatedExecute config options on Windows when in daemon-mode so it can handle multiple arguments. Patch courtesy of Zachary Murden.
- Fixed an error in the heuristic alert mechanism that would cause a single detection within an archive to alert once for every subsequent file scanned, potentially resulting in thousands of alerts for a single scan.
- Fixed clamd, clamav-milter, and freshclam to create PID files before dropping privileges, to avoid the possibility of an unprivileged user from changing the PID file so that a service manager will kill a different process. This change does make the services unable to clean up the PID file on exit.
- Fixed the false positive (.fp) signature feature. In prior versions, the hash in a false positive signature would be checked only against the current layer of a file being scanned. In 0.103, every file layer is hashed, and the hashes for each in the scan recursion list are checked. This ensures that .fp signatures containing a hash for any layer in the scan leading up to the alert will negate the alert.
- As an example, a hash for a zip containing the file which alerts would not prevent the detection in prior versions. Only the hash of the embedded file would work. For some file types where the outermost is always an archive, eg. docx files, this made .fp signatures next to useless. For certain file types where the scanned content was a normalized version of the original content, eg. HTML, the normalized version was never hashed and this meant that .fp signatures never worked.
- Fixed Trusted & Revoked Windows executable (PE) file signature rules (.crb) maximum functionality level (FLEVEL) which had been being treated as the minimum FLEVEL. These signatures enable ClamAV to trust executables that are digitally signed by trusted publishers or to alert on executables signed with compromised signing-certificates. The minimum and maximum FLEVELS enable or disable signatures at load time depending on the current ClamAV version.
- Fixed a bug wherein you could not build ClamAV with --enable-libclamav-only if curl was not installed on the system.
- Various other bug fixes, improvements and documentation improvements.

New Requirements:
- Autotools (automake, autoconf, m4, pkg-config, libtool) are now required to build from a Git clone because the files generated by these tools have been removed from the Git repository. To generate these files before you compile ClamAV, run autogen.sh. Users building with autotools from the release tarball should be unaffected.