1. Forum
  2. DOVE-Net
  3. Synchronet Discussion

  • Web server and BOT's

    From Richard Vonzel@VERT to All on Wed Jun 25 04:25:24 2025
    Hello All.

    I turned on the Synchronet Web server, works great, hsad my website running. Had my BBS's files available for access thru it. Yesterday I guess a BOT got on my site and jst downloaded all the file I had available. Here's the log entry for this,

    6/24 05:36:27p 3008 HTTP [20.171.207.62] Request 1825: GET /?page=001-forum.ssjs&sub=fsxnet_fsx_ads&thread=105 HTTP/1.1
    6/24 05:36:27p 3008 HTTP [20.171.207.62] User-Agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)
    6/24 05:36:28p 3008 HTTP [20.171.207.62] Request 1826: GET /?page=001-forum.ssjs&sub=fsxnet_fsx_gen&thread=177 HTTP/1.1
    6/24 05:36:28p 3008 HTTP [20.171.207.62] User-Agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)
    6/24 05:36:29p 3008 HTTP [20.171.207.62] Request 1827: GET /?page=001-forum.ssjs&sub=fsxnet_fsx_gen&thread=801 HTTP/1.1

    Is there a way to stop someone from using a BOT to take the files besides turning off the server?


    RickV

    --- SBBSecho 3.28-Win32
    * Origin: Live Wire BBS [livewirebbs.com:1025] (1:154/110)
    ■ Synchronet ■ Vertrauen ■ Home of Synchronet ■ [vert/cvs/bbs].synchro.net
  • From MRO@VERT/BBSESINF to Richard Vonzel on Wed Jun 25 14:02:22 2025
    Re: Web server and BOT's
    By: Richard Vonzel to All on Wed Jun 25 2025 04:25 am

    I turned on the Synchronet Web server, works great, hsad my website running. Had my BBS's files available for access thru it. Yesterday I guess a BOT got on my site and jst downloaded all the file I had available. Here's the log entry for this,

    6/24 05:36:27p 3008 HTTP [20.171.207.62] Request 1825: GET /?page=001-forum.ssjs&sub=fsxnet_fsx_ads&thread=105 HTTP/1.1
    6/24 05:36:27p 3008 HTTP [20.171.207.62] User-Agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)
    6/24 05:36:28p 3008 HTTP [20.171.207.62] Request 1826: GET /?page=001-forum.ssjs&sub=fsxnet_fsx_gen&thread=177 HTTP/1.1
    6/24 05:36:28p 3008 HTTP [20.171.207.62] User-Agent: Mozilla/5.0

    Is there a way to stop someone from using a BOT to take the files besides turning off the server?

    yeah there's a lot of ways. you can also just make it so logged in users have access to your msgs and files. please checkout the wiki and just read through it in your free time.

    The synchronet docs overall are a good source of information, unlike most other bbs softwares.

    you're running a service available to the entire internet. someone could go in and do some horrible stuff if you don't secure your system.
    ---
    ■ Synchronet ■ ::: BBSES.info - free BBS services :::
  • From poindexter FORTRAN@VERT/REALITY to Richard Vonzel on Wed Jun 25 08:11:39 2025
    Richard Vonzel wrote to All <=-

    Is there a way to stop someone from using a BOT to take the files
    besides turning off the server?

    Make all of your file content available only to authenticated users. I
    think my guest user has an access string of LEVEL 40, so I made all of
    my echo content LEVEL 50. If a guest hits the file areas, they won't see anything. If an authenticated user hits them, they'll see the areas.

    You could do something similar with the file echoes. Mine are wide open,
    I don't have anything I need to limit access to - although the bandwidth
    would be a pain to lose to bots.



    --- MultiMail/Win v0.52
    ■ Synchronet ■ .: realitycheckbbs.org :: scientia potentia est :.
  • From Digital Man@VERT to Richard Vonzel on Wed Jun 25 15:42:12 2025
    Re: Web server and BOT's
    By: Richard Vonzel to All on Wed Jun 25 2025 04:25 am

    Hello All.

    I turned on the Synchronet Web server, works great, hsad my website running. Had my BBS's files available for access thru it. Yesterday I guess a BOT got on my site and jst downloaded all the file I had available. Here's the log entry for this,

    6/24 05:36:27p 3008 HTTP [20.171.207.62] Request 1825: GET /?page=001-forum.ssjs&sub=fsxnet_fsx_ads&thread=105 HTTP/1.1
    6/24 05:36:27p 3008 HTTP [20.171.207.62] User-Agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)
    6/24 05:36:28p 3008 HTTP [20.171.207.62] Request 1826: GET /?page=001-forum.ssjs&sub=fsxnet_fsx_gen&thread=177 HTTP/1.1
    6/24 05:36:28p 3008 HTTP [20.171.207.62] User-Agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)
    6/24 05:36:29p 3008 HTTP [20.171.207.62] Request 1827: GET /?page=001-forum.ssjs&sub=fsxnet_fsx_gen&thread=801 HTTP/1.1

    Is there a way to stop someone from using a BOT to take the files besides turning off the server?

    You can disable the Files tab in ecWebv4 if I recall correcctly.
    --
    digital man (rob)

    Breaking Bad quote #16:
    Thinking Operation Breath Mint evertime you and me are on a stakeout together. Norco, CA WX: 83.1°F, 46.0% humidity, 15 mph W wind, 0.00 inches rain/24hrs
    ---
    ■ Synchronet ■ Vertrauen ■ Home of Synchronet ■ [vert/cvs/bbs].synchro.net
  • From nelgin@VERT/EOTLBBS to Richard Vonzel on Wed Jun 25 23:26:42 2025
    Re: Web server and BOT's
    By: Richard Vonzel to All on Wed Jun 25 2025 04:25:24

    I turned on the Synchronet Web server, works great, hsad my website running. Had my BBS's files available for access thru it. Yesterday I guess a BOT got on my site and jst downloaded all the file I had available. Here's the log entry for this,

    I have found gptbot ignores a global disallow rule in robots.txt

    User-agent: *
    Disallow: /

    But you could try adding

    User-agent: gptbot
    Disallow :/

    It could take a few days but you'll see when they start to pickup your robots.txt file.

    Another option would be to just block the ip ranges altogether.

    Another option would be to block the published subnets. I have these blocked using ipsets

    4.227.36.0/25
    20.171.207.0/24
    52.230.152.0/24
    172.182.204.0/24
    20.171.206.0/24
    172.182.214.0/24
    20.125.66.80/28
    172.182.215.0/24

    That seems to be the easiest way and just don't deal with them at all.

    ---
    ■ Synchronet ■ End Of The Line BBS - endofthelinebbs.com
  • From Mrsteve@VERT to nelgin on Thu Jun 26 09:51:55 2025
    not able to up/download QWK message packet
    *?

    I did receive a Fidonews file..

    ---
    ■ Synchronet ■ Vertrauen ■ Home of Synchronet ■ [vert/cvs/bbs].synchro.net
  • From nelgin@VERT/EOTLBBS to Mrsteve on Thu Jun 26 12:25:15 2025
    Re: Web server and BOT's
    By: Mrsteve to nelgin on Thu Jun 26 2025 09:51:55

    not able to up/download QWK message packet
    *?

    I did receive a Fidonews file..

    That's nice but what does it have to do with web server bots?

    ---
    ■ Synchronet ■ End Of The Line BBS - endofthelinebbs.com
  • From MRO@VERT/BBSESINF to nelgin on Thu Jun 26 15:24:39 2025
    Re: Web server and BOT's
    By: nelgin to Richard Vonzel on Wed Jun 25 2025 11:26 pm

    guess a BOT got on my site and jst downloaded all the file I had
    available. Here's the log entry for this,

    I have found gptbot ignores a global disallow rule in robots.txt

    User-agent: *
    Disallow: /

    But you could try adding


    dude where have you been? most of those bots ignore robots.txt

    google bot, archive.org's bots, all of them. there's no law that they
    have to obey robots.txt
    ---
    ■ Synchronet ■ ::: BBSES.info - free BBS services :::