• qwk errors / libarchive

    From Fernando Toledo@VERT to GitLab issue in main/sbbs on Wed Jun 9 03:55:04 2021
    open https://gitlab.synchro.net/main/sbbs/-/issues/272

    Hi i'm setup my sbbs as qwk network hub

    I found two issues:

    1) A mystic node that connect to my system to get the packet, get a libarchive error when making the qwk file at sbbs log

    ```
    Jun 9 00:23:52 scarlet synchronet: evnt packQWK QWK pack semaphore signaled: /sbbs/data/pack0195.now
    Jun 9 00:23:52 scarlet synchronet: evnt packQWK <DCKMYS> Packing QWK Message Packet
    Jun 9 00:23:52 scarlet synchronet: evnt packQWK <DCKMYS> scanned 3 sub-boards for new messages
    Jun 9 00:23:52 scarlet synchronet: evnt packQWK <DCKMYS> packed 1 messages (0.5K bytes) in 1 seconds (1 msgs/sec)
    Jun 9 00:23:52 scarlet synchronet: evnt packQWK <DCKMYS> libarchive error (archive_write_header returned -30) creating /sbbs/data/file/0195.qwk
    Jun 9 00:23:52 scarlet synchronet: evnt packQWK <DCKMYS> Packing completed: /sbbs/data/file/0195.qwk

    ```
    Also te mystic log show a invalid zip file error when process it:
    ```
    QWKPOLL Startup (v1.12 A47 Linux/64 Compiled 2021/06/02 09:48:24)

    Exchanging Mail for Zudaka Net
    Exported @docksud.rep -> 0 msgs
    Connecting via FTP to bbs.docksud.com.ar
    Connected
    Local IP is xx.xx.xx.xx
    Logged in as dckmys
    Downloading QWK packet
    OK: docksud.qwk (5,731 bytes)
    error: invalid zip file with overlapped components (possible zip bomb) Importing QWK packet:
    Imported 2 messages (0 failed)

    ```

    2) The mystic sems to add a qlr.dat file to .rep packet.. i can't find any reference to this file
    and sbbs raise a error that file exists when trying to copy (i running as sbbs root so i asume no permission issues)

    ```
    Jun 9 00:26:52 scarlet synchronet: evnt unpackREP Inbound QWK Reply Packet detected: /sbbs/data/file/0196.rep
    Jun 9 00:26:52 scarlet synchronet: evnt unpackREP <EPI> libarchive extracted 2 files from /sbbs/data/file/0196.rep
    Jun 9 00:26:52 scarlet synchronet: evnt unpackREP <EPI> posted QWK message on Zudaka Net Area para pruebas
    Jun 9 00:26:52 scarlet synchronet: evnt unpackREP <EPI> !ERROR 17 (File exists) in main.cpp line 3963 (mv) CopyFile "/sbbs/temp/event/qlr.dat" access=0 info=/sbbs/data/qnet/EPI.in/qlr.dat
    Jun 9 00:26:52 scarlet synchronet: evnt unpackREP <EPI> Received file: qlr.dat Jun 9 00:26:52 scarlet synchronet: evnt unpackREP <EPI> Unpacking completed: /sbbs/data/file/0196.rep (1 msgs, 0 errors, 0 dupes)
    Jun 9 00:26:52 scarlet synchronet: evnt packQWK QWK pack semaphore signaled: /sbbs/data/pack0196.now
    ```

    Although these errors exist, messages seem to arrive from one system to another But I'm not completely sure if everything is okay.
    thanks!

    Saludos!

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Fernando Toledo@VERT to GitLab note in main/sbbs on Wed Jun 9 03:57:46 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2019

    I forgot:

    my system is Debian stretch and libarchive13 version 3.2.2-2+deb9u2

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Fernando Toledo@VERT to GitLab note in main/sbbs on Wed Jun 9 04:02:06 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2020

    Can I go back and use the extractable file types option for .ZIP files option (using zip/unzip) untis this issue get fixed?

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Fernando Toledo@VERT to GitLab note in main/sbbs on Wed Jun 9 04:11:28 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2021

    I am doing tests and I noticed that problem 1 only happened if the user is a qwk account (REST Q), it does not happen for a common users

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Rob Swindell@VERT to GitLab note in main/sbbs on Wed Jun 9 04:45:59 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2022

    Please enter separate issues here for separate problems - don't combine multiple problems into a single issue.

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Rob Swindell@VERT to GitLab note in main/sbbs on Wed Jun 9 04:47:49 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2023

    As for the first issue `libarchive error (archive_write_header returned -30) creating`, what is that user's temp/QWK archive type set to?

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Rob Swindell@VERT to GitLab issue in main/sbbs on Wed Jun 9 04:59:03 2021
    close https://gitlab.synchro.net/main/sbbs/-/issues/272

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Rob Swindell@VERT to GitLab note in main/sbbs on Wed Jun 9 05:22:53 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2024

    I was able to reproduce a similar error to the first reported error, but only when the archive type is set to "tbz" and only on Windows (works fine on my Debian Linux system, but maybe because I have bzip2 installed):
    libarchive error (archive_write_open_filename(s:\sbbs\node2\temp\VERT.qwk) returned -30: Can't launch external program: bzip2) creating s:\sbbs\node2\temp\VERT.qwk

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Fernando Toledo@VERT to GitLab note in main/sbbs on Wed Jun 9 05:40:33 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2025

    Sorry! i think that these issueas was related to qwk accounts. My apologies

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Rob Swindell@VERT to GitLab note in main/sbbs on Wed Jun 9 05:49:43 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2026

    As for the QLR.DAT file, I have no idea what that file is supposed to be used for either (not mentioned in any of my QWK specs). I just add it (qlr.dat) to my text/file.can, so it gets ignored whenever I receive one from a Mystic system.

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Fernando Toledo@VERT to GitLab note in main/sbbs on Wed Jun 9 05:53:22 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2027

    File type are zip, i dont found any option to set default qwk file type for new users at scfg.

    I think that sbbs asume the first item from Compressible File Types or just zip:

    ![image](/uploads/7c0f0ed8ca22f8e580d45a4a75b42287/image.png)

    The user qwk settings:

    ![image](/uploads/34eeab2f23911c0c8d1a8cdaaac8c7f5/image.png)

    i dont have tbz o tgz setup at scfg.. the are come from libarchive, true?

    ![image](/uploads/da92cc77341cc04e7eea170e5d4bb436/image.png)

    also i notice that post select the zip option 1 now are in lower case:

    ![image](/uploads/0328e946e5c2ec89588036cbacab3c15/image.png)

    so maybe the mixes case ZIP / zip can raise the issue?

    thanks!

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Rob Swindell@VERT to GitLab note in main/sbbs on Wed Jun 9 05:56:06 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2028

    The case should not matter: libarchive can create .ZIP or .zip files.

    New user accounts have their temp/QWK archive type set to "ZIP" by default.

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Fernando Toledo@VERT to GitLab note in main/sbbs on Wed Jun 9 06:00:10 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2029

    I have bzip2 installed on the system and user config is zip, so should not use bzip compression i think.

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Fernando Toledo@VERT to GitLab note in main/sbbs on Wed Jun 9 06:00:33 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2030

    ok! i will add to file.can thanks

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Rob Swindell@VERT to GitLab note in main/sbbs on Wed Jun 9 06:03:21 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2031

    I just added more details for any `archive_write_header` failures. So if you could update an reproduce the error and provide those details, that be helpful.

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Rob Swindell@VERT to GitLab note in main/sbbs on Wed Jun 9 06:04:23 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2032

    Yup. Sounds like libarchive is failing to create a ZIP file. Why it would only fail for some users, I have no idea.

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Fernando Toledo@VERT to GitLab note in main/sbbs on Wed Jun 9 06:12:13 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2033

    ```
    Jun 9 03:05:32 scarlet synchronet: ftp 0082 CTRL thread started
    Jun 9 03:05:32 scarlet synchronet: ftp 0082 CTRL connection accepted from: 186.189.236.69 port 46822
    Jun 9 03:05:32 scarlet synchronet: ftp 0082 <DCKMYS> logged in (27 today, 92 total)
    Jun 9 03:05:32 scarlet synchronet: ftp 0082 <DCKMYS> creating QWK packet... Jun 9 03:05:53 scarlet synchronet: ftp 0090 CTRL thread started
    Jun 9 03:05:53 scarlet synchronet: ftp 0090 CTRL connection accepted from: 10.5.1.2 port 56848
    Jun 9 03:05:53 scarlet synchronet: ftp 0090 CTRL thread terminated (1 clients and 2 threads remain, 5 served)
    Jun 9 03:06:17 scarlet synchronet: evnt packQWK QWK pack semaphore signaled: /sbbs/data/pack0195.now
    Jun 9 03:06:17 scarlet synchronet: evnt packQWK <DCKMYS> Packing QWK Message Packet
    Jun 9 03:06:17 scarlet synchronet: evnt packQWK <DCKMYS> scanned 5 sub-boards for new messages
    Jun 9 03:06:17 scarlet synchronet: evnt packQWK <DCKMYS> packed 1 messages (0.6K bytes) in 1 seconds (1 msgs/sec)
    Jun 9 03:06:17 scarlet synchronet: evnt packQWK <DCKMYS> libarchive error (archive_write_header returned -30: (null)) creating /sbbs/data/file/0195.qwk
    Jun 9 03:06:17 scarlet synchronet: evnt packQWK <DCKMYS> Packing completed: /sbbs/data/file/0195.qwk
    Jun 9 03:06:17 scarlet synchronet: ftp 0082 <DCKMYS> downloading QWK packet (5642 bytes) in passive mode
    Jun 9 03:06:17 scarlet synchronet: ftp 0082 <DCKMYS> DATA Transfer successful: 5642 bytes sent in 0 seconds (11284 cps)
    Jun 9 03:06:17 scarlet synchronet: ftp 0082 <DCKMYS> logged off
    Jun 9 03:06:17 scarlet synchronet: ftp 0082 CTRL thread terminated (0 clients and 1 threads remain, 5 served)
    ```

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Fernando Toledo@VERT to GitLab note in main/sbbs on Wed Jun 9 06:17:11 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2034

    any chance to back to use zip/unzip settings again? at least until the source of the problem can be determined

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Rob Swindell@VERT to GitLab note in main/sbbs on Wed Jun 9 06:19:45 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2035

    Well that sucks.

    Can you use 'jsexec archive create' to create a ZIP file? For example:

    ```
    $ jsexec archive create /tmp/test.zip /tmp/*.txt

    JSexec v3.19a-Linux master/fc7e041fc Debug - Execute Synchronet JavaScript Module
    Compiled Jun 8 2021 22:05:56 with GCC 8.3.0

    Loading configuration files from /sbbs/ctrl
    JavaScript-C 1.8.5 2011-03-31
    JavaScript: Creating runtime: 20971520 bytes

    Reading script from /sbbs/exec/archive.js
    /sbbs/exec/archive.js compiled in 0.00 seconds
    1 files archived
    ```

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Fernando Toledo@VERT to GitLab note in main/sbbs on Wed Jun 9 06:20:23 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2036

    notice the description that zip format say "read only, with some limitations, uses zlib"

    ```
    root@scarlet:/sbbs/exec# apt-cache show libarchive13
    Package: libarchive13
    Source: libarchive
    Version: 3.2.2-2+deb9u2
    Installed-Size: 774
    Maintainer: Debian Libarchive Maintainers <ah-libarchive@debian.org> Architecture: amd64
    Depends: libacl1 (>= 2.2.51-8), libbz2-1.0, libc6 (>= 2.16), liblz4-1 (>= 0.0~r130), liblzma5 (>= 5.2.2), liblzo2-2, libnettle6, libxml2 (>= 2.7.4), zlib1g (>= 1:1.1.4)
    Suggests: lrzip
    Description-en: Multi-format archive and compression library (shared library)
    The libarchive library provides a flexible interface for reading and writing
    archives in various formats such as tar and cpio. libarchive also supports
    reading and writing archives compressed using various compression filters such
    as gzip and bzip2. The library is inherently stream-oriented; readers serially
    iterate through the archive, writers serially add things to the archive.
    .
    Archive formats supported are:
    .
    * tar (read and write, including GNU extensions)
    * pax (read and write, including GNU and star extensions)
    * cpio (read and write, including odc and newc variants)
    * iso9660 (read and write, including Joliet and Rockridge extensions, with
    some limitations)
    * zip (read only, with some limitations, uses zlib)
    * mtree (read and write)
    * shar (write only)
    * ar (read and write, including BSD and GNU/SysV variants)
    * empty (read only; in particular, note that no other format will accept an
    empty file)
    * raw (read only)
    * xar (read only)
    * rar (read only, with some limitations)
    * 7zip (read and write, with some limitations)
    .
    Filters supported are:
    .
    * gzip (read and write, uses zlib)
    * bzip2 (read and write, uses bzlib)
    * compress (read and write, uses an internal implementation)
    * uudecode (read only)
    * separate command-line compressors with fixed-signature auto-detection
    * xz and lzma (read and write using liblzma)
    .
    This package provides the libarchive shared library.
    Description-md5: 3efebf57a3d06179c6d5746db080711f
    Multi-Arch: same
    Homepage: http://www.libarchive.org/
    Tag: role::shared-lib
    Section: libs
    Priority: optional
    Filename: pool/main/liba/libarchive/libarchive13_3.2.2-2+deb9u2_amd64.deb
    Size: 294448
    MD5sum: 113b3c77663c77f55ad8d659736884f4
    SHA256: d008342111fd7899807509013148eeb2379b432689790c1d701df1911e0c3fc9

    ```

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Rob Swindell@VERT to GitLab note in main/sbbs on Wed Jun 9 06:24:27 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2037

    Yeah, I see that, but it works fine with libarchive Version: 3.3.3-4+deb10u1

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Rob Swindell@VERT to GitLab note in main/sbbs on Wed Jun 9 06:26:07 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2038

    And https://github.com/libarchive/libarchive/wiki/LibarchiveFormats has zip as writable (and it is).

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Fernando Toledo@VERT to GitLab note in main/sbbs on Wed Jun 9 06:29:25 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2039

    ```
    root@scarlet:/sbbs/exec# echo 1 >/tmp/pepe.txt
    root@scarlet:/sbbs/exec# jsexec archive.js create /tmp/pepe.zip /tmp/pepe.txt

    JSexec v3.19a-Linux master/a2110bfca Debug - Execute Synchronet JavaScript Module
    Compiled Jun 9 2021 03:02:41 with GCC 6.3.0

    Loading configuration files from /sbbs/ctrl
    JavaScript-C 1.8.5 2011-03-31
    JavaScript: Creating runtime: 16777216 bytes

    Reading script from /sbbs/exec/archive.js
    /sbbs/exec/archive.js compiled in 0.00 seconds
    1 files archived
    /sbbs/exec/archive.js executed in 0.00 seconds

    JavaScript: Destroying context
    JavaScript: Destroying runtime
    root@scarlet:/sbbs/exec# ls -l /tmp/pepe.zip
    -rw-r--r-- 1 root root 182 jun 9 03:27 /tmp/pepe.zip
    root@scarlet:/sbbs/exec# file /tmp/pepe.zip
    /tmp/pepe.zip: Zip archive data, at least v2.0 to extract root@scarlet:/sbbs/exec# unzip -l /tmp/pepe.zip
    Archive: /tmp/pepe.zip
    Length Date Time Name
    --------- ---------- ----- ----
    2 2021-06-09 03:27 pepe.txt
    --------- -------
    2 1 file

    ```

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Fernando Toledo@VERT to GitLab note in main/sbbs on Wed Jun 9 06:37:48 2021
    https://gitlab.synchro.net/main/sbbs/-/issues/272#note_2040

    just for try to insolate it, the issue occurs on the context from ftp connection for download the qwk packet

    qnet events work fine:

    ```
    root@scarlet:/sbbs/exec# grep -i libarchive /var/log/sbbs.log
    Jun 8 07:04:47 scarlet synchronet: evnt QNET libarchive extracted 9 files from /sbbs/data/VERT.qwk
    Jun 8 08:04:50 scarlet synchronet: evnt QNET libarchive extracted 8 files from /sbbs/data/VERT.qwk
    Jun 8 09:05:59 scarlet synchronet: evnt QNET libarchive extracted 8 files from /sbbs/data/VERT.qwk
    Jun 8 10:05:30 scarlet synchronet: evnt QNET libarchive extracted 11 files from /sbbs/data/VERT.qwk
    Jun 8 12:05:07 scarlet synchronet: evnt QNET libarchive extracted 9 files from /sbbs/data/VERT.qwk
    Jun 8 13:04:57 scarlet synchronet: evnt QNET libarchive extracted 9 files from /sbbs/data/VERT.qwk

    ```

    process rep uploaded by ftp work fine(from qwk account):

    ```
    Jun 9 00:13:52 scarlet synchronet: evnt unpackREP <EPI> libarchive extracted 2 files from /sbbs/data/file/0196.rep
    ```

    generate the qwk packet from terminal (logged user / qwk account) work fine:

    ```
    Jun 9 02:56:43 scarlet synchronet: term Node 1 <DCKMYS> libarchive created /sbbs/nodes/node1/temp/DOCKSUD.qwk from 5 files

    ```

    sems to insolated to packQWK event when it was trigger on ftp incomming call

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net