• DUPES!

    From Roger Nelson@1:3828/7 to All on Tue Oct 25 21:30:40 2016

    ┌PATH INFORMATION
    │ │
    │ THERE Houma LA 1:3828/7 │
    │ · Saskatchewan 1:140/1 │
    │ · Slaterville Springs NY 1:261/38 │
    │ · Lafayette LA 1:393/68 │
    │ · " 1:770/1 │
    │ · " 1:280/464 │
    │ · Anderson IN 1:227/51 │
    │ · Lansing MI 1:120/544 │
    │ · Saskatchewan 1:140/1 │
    │ HERE! Houma LA 1:3828/7 │
    │ │

    Was this really necessary?


    Regards,

    Roger

    --- DB 3.99 + W10 (1607)
    * Origin: NCS BBS - Houma, LoUiSiAna (1:3828/7)
  • From Paul Quinn@3:640/384 to Roger Nelson on Wed Oct 26 13:19:53 2016
    Hi! Roger,

    On 25 Oct 16 21:30, you wrote to All:

    ┌PATH INFORMATION

    [ ...trimmed... ]

    Was this really necessary?

    You get that effect with messages flitting in & out of zones, and SEENBYs being
    stripped. It's the Confucious paradigm: stuff happens.

    Cheers,
    Paul.

    ... "In my jungle you would be just another asshole", Jamie Shannon 1980.
    --- Paul's Win98SE VirtualBox
    * Origin: Quinn's Post - Maryborough, Queensland, OZ (3:640/384)
  • From Wilfred van Velzen@2:280/464 to Roger Nelson on Wed Oct 26 09:21:09 2016
    Hi Roger,

    On 2016-10-25 21:30:40, you wrote to All:

    ┌PATH INFORMATION
    │ │
    │ THERE Houma LA 1:3828/7 │
    │ · Saskatchewan 1:140/1 │
    │ · Slaterville Springs NY 1:261/38 │
    │ · Lafayette LA 1:393/68 │
    │ · " 1:770/1 │
    │ · " 1:280/464 │
    │ · Anderson IN 1:227/51 │
    │ · Lansing MI 1:120/544 │
    │ · Saskatchewan 1:140/1 │
    │ HERE! Houma LA 1:3828/7 │
    ^

    Your software is Z1 biased...

    Was this really necessary?

    Some system in that path is probably stripping seen-by's.

    If your system detected it as dupe, don't worry about it. If it didn't get your
    dupe detection fixed. ;)

    But why did 140/1 not detect it as dupe and send it back to you?

    Bye, Wilfred.

    --- FMail-W32 1.73.0.18-B20161025
    * Origin: FMail development HQ (2:280/464)
  • From Roger Nelson@1:3828/7 to Paul Quinn on Wed Oct 26 06:53:47 2016

    Hi,

    On 25 Oct 16 21:30, you wrote to All:

    ┌PATH INFORMATION

    [ ...trimmed... ]

    Was this really necessary?

    You get that effect with messages flitting in & out of zones, and SEENBYs being stripped. It's the Confucious paradigm: stuff happens.

    Yes, I'm aware of that, but I see it in so many echoes I receive and some dudes
    want to blame it on my tosser. That just isn't the case, so I take the advice of those guys with a grain of salt (according to doctor, even a grain is too much). (-:

    ...Go bland and go blind!


    Regards,

    Roger

    --- DB 3.99 + W10 (1607)
    * Origin: NCS BBS - Houma, LoUiSiAna (1:3828/7)
  • From mark lewis@1:3634/12.73 to Roger Nelson on Wed Oct 26 11:53:56 2016

    26 Oct 16 06:53, you wrote to Paul Quinn:

    ┌PATH INFORMATION

    [ ...trimmed... ]

    Was this really necessary?

    You get that effect with messages flitting in & out of zones, and
    SEENBYs being stripped. It's the Confucious paradigm: stuff happens.

    Yes, I'm aware of that, but I see it in so many echoes I receive and some dudes want to blame it on my tosser. That just isn't the case, so I take the advice of those guys with a grain of salt (according to doctor, even a grain is too much). (-:

    does your tosser toss them into your badmessages or dupes area OR does it toss them into the echo they are supposed to be in?

    if your tosser tosses them into your badmessages or dupes area, then it has caught them as a duplicate... otherwise, it has missed that they are a dupe (unless your tosser has a setting to keep them in the original echo)...

    in most cases, the system ""above"" you that sends the dupes to you should catch them... it may not be able to do so if its dupe database gets flushed of dupe data real quickly because of the amount of traffic it handles... that's if
    that system even has dupe detection enabled... they may not have it enabled if they elect to send all mail on to their connections and let them determine if it is a dupe or not... they may decide to do that so as to avoid false positives being stopped and not propogating...

    )\/(ark

    Always Mount a Scratch Monkey
    Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
    ... At least this isn't a bloody French tagline.
    ---
    * Origin: (1:3634/12.73)
  • From Roger Nelson@1:3828/7 to mark lewis on Fri Oct 28 11:30:29 2016
    On Wed Oct-26-2016 11:53, mark lewis (1:3634/12.73) wrote to Roger Nelson:

    26 Oct 16 06:53, you wrote to Paul Quinn:

    [...]

    does your tosser toss them into your badmessages or dupes area OR
    does it toss them into the echo they are supposed to be in?

    Both, depending on the path.

    if your tosser tosses them into your badmessages or dupes area,
    then it has caught them as a duplicate... otherwise, it has missed
    that they are a dupe (unless your tosser has a setting to keep them
    in the original echo)...

    Bad_Msgs area.

    in most cases, the system ""above"" you that sends the dupes to you
    should catch them... it may not be able to do so if its dupe
    database gets flushed of dupe data real quickly because of the
    amount of traffic it handles... that's if that system even has dupe detection enabled... they may not have it enabled if they elect to
    send all mail on to their connections and let them determine if it
    is a dupe or not... they may decide to do that so as to avoid false positives being stopped and not propogating...

    But that system's dupe detector worked well for years and years until lately. No, I believe the problem lies in a misconfiguration somewhere eles.


    Regards,

    Roger
    --- timEd/386 1.10.y2k+ W10 (1607)
    * Origin: NCS BBS - Houma, LoUiSiAna - (1:3828/7)
  • From Wilfred van Velzen@2:280/464 to Roger Nelson on Fri Oct 28 19:32:30 2016
    Hi,

    On 2016-10-28 11:30:29, Roger Nelson wrote to mark lewis:
    about: "DUPES!":

    But that system's dupe detector worked well for years and years until lately. No, I believe the problem lies in a misconfiguration somewhere eles.

    Are the message date/time the same of the original and dupe message? (See the discussion in FN_SYSOP)

    Bye, Wilfred.


    --- FMail-W32 1.73.1.20-B20161028
    * Origin: Native IPv6 connectable node (2:280/464)
  • From mark lewis@1:3634/12.73 to Roger Nelson on Fri Oct 28 15:14:40 2016

    28 Oct 16 11:30, you wrote to me:

    in most cases, the system ""above"" you that sends the dupes to you
    should catch them... it may not be able to do so if its dupe database
    gets flushed of dupe data real quickly because of the amount of
    traffic it handles... that's if that system even has dupe detection
    enabled... they may not have it enabled if they elect to send all
    mail on to their connections and let them determine if it is a dupe
    or not... they may decide to do that so as to avoid false positives
    being stopped and not propogating...

    But that system's dupe detector worked well for years and years until lately. No, I believe the problem lies in a misconfiguration somewhere eles.

    ok... i see what you are saying now... it is no misconfiguration, though... it is a change in the landscape... previously there were fewer dupes to deal with... they were part of a system that had only two others in the fully connected polygon... today, there are many more systems in numerous polygons, fully connected and not so fully connected... today's game is about redundancy and there is no cost to transport dupes like there was in the old POTS dialup days of yor... with more dupes flying around, old school slash old style dupe databases can get overrun pretty quickly... this is especially true in older 16
    and 32 bit code... 16bit moreso, of course... there is a limit to the sheer number of values they can store and there is also a filesize limitation... the filesize limitation used to be OS dependant but older software on new OSes will
    find their index counters to be the limitation on file sizes...

    we're not even touching on some of the new ""slightly"" buggy software that is available today, either...

    )\/(ark

    Always Mount a Scratch Monkey
    Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
    ... On Nytol: Warning: may cause drowsiness.
    ---
    * Origin: (1:3634/12.73)
  • From Wilfred van Velzen@2:280/464 to mark lewis on Fri Oct 28 23:29:23 2016
    Hi,

    On 2016-10-28 15:14:40, mark lewis wrote to Roger Nelson:
    about: "DUPES!":

    with more dupes flying around, old school slash old style dupe
    databases can get overrun pretty quickly...

    Eh... You are only storing a checksum of the first original. More dupes doesn't
    mean more space needed in the dupe database! ;)


    Bye, Wilfred.


    --- FMail-W32 1.73.1.20-B20161028
    * Origin: Native IPv6 connectable node (2:280/464)
  • From mark lewis@1:3634/12.73 to Wilfred van Velzen on Fri Oct 28 19:14:34 2016

    28 Oct 16 23:29, you wrote to me:

    with more dupes flying around, old school slash old style dupe
    databases can get overrun pretty quickly...

    Eh... You are only storing a checksum of the first original. More
    dupes doesn't mean more space needed in the dupe database! ;)

    that depends on the database format ;)

    )\/(ark

    Always Mount a Scratch Monkey
    Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
    ... Famous Last Words: "Go away! I'm all right!" - H G Wells
    ---
    * Origin: (1:3634/12.73)
  • From Wilfred van Velzen@2:280/464 to mark lewis on Mon Oct 31 13:06:31 2016
    Hi mark,

    On 2016-10-28 19:14:34, you wrote to me:

    with more dupes flying around, old school slash old style dupe
    databases can get overrun pretty quickly...

    Eh... You are only storing a checksum of the first original. More
    dupes doesn't mean more space needed in the dupe database! ;)

    that depends on the database format ;)

    It shouldn't!

    Dupes have by definition the same dupe-checksum as the original, so storing them is useless!

    Bye, Wilfred.

    --- FMail-W32 1.73.1.22-B20161030
    * Origin: FMail development HQ (2:280/464)
  • From mark lewis@1:3634/12.73 to Wilfred van Velzen on Mon Oct 31 13:26:38 2016
    * Originally in bama
    * Crossposted in fidosoft.husky

    31 Oct 16 13:06, you wrote to me:

    with more dupes flying around, old school slash old style dupe
    databases can get overrun pretty quickly...

    Eh... You are only storing a checksum of the first original. More
    dupes doesn't mean more space needed in the dupe database! ;)

    that depends on the database format ;)

    It shouldn't!

    again, that depends on the database format...

    Dupes have by definition the same dupe-checksum as the original, so storing them is useless!

    maybe to you but to another developer who may be doing other things than /just/
    dupes, it may be needed or desired... don't be so square that your mind is closed to other possibilities and thought processes... perhaps someone is keeping count of duplicate MSGIDs and needing also to store the CRC/hash for each of them...

    example: yesterday i saw two messages with exactly the same MSGID, header, time
    stamps, and what looks to be the exact same message body... the seenbys and paths were different yet HPT with its HASH+MSGID dupe checking missed seeing the second one as a dupe... both were back-to-back in my message base so it was
    easy to flip back and forth between them to try to see any differences... golded+'s [I]nfo showed that they were virtually identical but there looked to be one space character immediately after the MSGID that was in one post and not
    the other... now, get this, both MSGIDs, even though they are the same, are in the dupe database but with different hashes...

    )\/(ark

    Always Mount a Scratch Monkey
    Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
    ... We have no free press any more. It is bought and paid for.
    ---
    * Origin: (1:3634/12.73)
  • From Wilfred van Velzen@2:280/464 to mark lewis on Mon Oct 31 21:21:37 2016
    Hi,

    On 2016-10-31 13:26:38, mark lewis wrote to Wilfred van Velzen:
    about: "DUPES!":

    Dupes have by definition the same dupe-checksum as the original, so
    storing them is useless!

    maybe to you but to another developer who may be doing other things than /just/ dupes, it may be needed or desired... don't be so square that your mind is closed to other possibilities and thought processes... perhaps someone is keeping count of duplicate MSGIDs and needing also to store the CRC/hash for each of them...

    If the CRC/hash is the same, because it's a dupe, you don't have to store it again.

    If you want to keep count of duplicate MSGID's you store those, or just a count
    of them. That's got little to do with dupe hash's...

    example: yesterday i saw two messages with exactly the same MSGID,
    header, time stamps, and what looks to be the exact same message
    body... the seenbys and paths were different yet HPT with its
    HASH+MSGID dupe checking missed seeing the second one as a dupe...
    both were back-to-back in my message base so it was easy to flip back
    and forth between them to try to see any differences... golded+'s
    [I]nfo showed that they were virtually identical but there looked to
    be one space character immediately after the MSGID that was in one
    post and not the other... now, get this, both MSGIDs, even though they
    are the same, are in the dupe database but with different hashes...

    I fail to see what's your point here. They weren't dupes according to hpt, so the different crc/hash's were stored.

    Bye, Wilfred.


    --- FMail-W32 1.73.1.23-B20161031
    * Origin: Native IPv6 connectable node (2:280/464)
  • From mark lewis@1:3634/12.73 to Wilfred van Velzen on Mon Oct 31 22:33:26 2016
    * Originally in bama
    * Crossposted in fidosoft.husky

    31 Oct 16 21:21, you wrote to me:

    Dupes have by definition the same dupe-checksum as the original, so
    storing them is useless!

    maybe to you but to another developer who may be doing other things
    than /just/ dupes, it may be needed or desired... don't be so square
    that your mind is closed to other possibilities and thought
    processes... perhaps someone is keeping count of duplicate MSGIDs and
    needing also to store the CRC/hash for each of them...

    If the CRC/hash is the same, because it's a dupe, you don't have to
    store it again.

    like i said before, that depends on the database format...

    If you want to keep count of duplicate MSGID's you store those, or
    just a count of them. That's got little to do with dupe hash's...

    they can easily be stored and counted in the same database... who are you or i to say that that is wrong? ;)

    example: yesterday i saw two messages with exactly the same MSGID,
    header, time stamps, and what looks to be the exact same message
    body... the seenbys and paths were different yet HPT with its
    HASH+MSGID dupe checking missed seeing the second one as a dupe...
    both were back-to-back in my message base so it was easy to flip back
    and forth between them to try to see any differences... golded+'s
    [I]nfo showed that they were virtually identical but there looked to
    be one space character immediately after the MSGID that was in one
    post and not the other... now, get this, both MSGIDs, even though
    they are the same, are in the dupe database but with different
    hashes...

    I fail to see what's your point here. They weren't dupes according to
    hpt, so the different crc/hash's were stored.

    the point is that they ARE dupes and should have been detected as dupes... someone's formula is incorrect... it reminds me of a mailer/tosser combo that takes the header plus the first 40 bytes to calculate their hash with... software developers need to know about this so they can take steps to put the data in the proper place so this particular software will see the data and catch dupes as needed... this means to place the MSGID at the top or just after
    the AREA tag... but see? this is the type of knowledge that is gained after years of working with FTN software and being aware of how things were done in the past... not assuming that everything is done the same everywhere... especially since many developers are now gone and their knowlegede is no longer
    first or second hand... even with new developers having taken over older software and the details of the operation not having been transmitted along with the actual code base... the new maintainers have to figure it out for themselves or be told by someone that knows these details... reading the code is one thing... understanding it and why it was done a certain way is another...

    )\/(ark

    Always Mount a Scratch Monkey
    Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
    ... I'm not an asshole. I'm a hemorrhoid. I irritate assholes.
    ---
    * Origin: (1:3634/12.73)
  • From Roger Nelson@1:3828/7 to mark lewis on Tue Nov 1 15:21:22 2016
    On Mon Oct-31-2016 22:33, mark lewis (1:3634/12.73) wrote to Wilfred van Velzen:

    @REPLY: 2:280/464 5817a8b8
    @MSGID: 1:3634/12.73 58180133
    @PID: GED+LNX 1.1.5-b20150715
    @CHRS: CP437 2
    @TZUTC: -0400
    @TID: hpt/lnx 1.9.0-cur 07-09-15
    * Originally in bama
    * Crossposted in fidosoft.husky

    31 Oct 16 21:21, you wrote to me:

    Dupes have by definition the same dupe-checksum as the original, so
    storing them is useless!

    maybe to you but to another developer who may be doing other things
    than /just/ dupes, it may be needed or desired... don't be so square
    that your mind is closed to other possibilities and thought
    processes... perhaps someone is keeping count of duplicate MSGIDs and
    needing also to store the CRC/hash for each of them...

    If the CRC/hash is the same, because it's a dupe, you don't have to
    store it again.

    like i said before, that depends on the database format...

    If you want to keep count of duplicate MSGID's you store those, or
    just a count of them. That's got little to do with dupe hash's...

    they can easily be stored and counted in the same database... who
    are you or i to say that that is wrong? ;)

    example: yesterday i saw two messages with exactly the same MSGID,
    header, time stamps, and what looks to be the exact same message
    body... the seenbys and paths were different yet HPT with its
    HASH+MSGID dupe checking missed seeing the second one as a dupe...
    both were back-to-back in my message base so it was easy to flip back
    and forth between them to try to see any differences... golded+'s
    [I]nfo showed that they were virtually identical but there looked to
    be one space character immediately after the MSGID that was in one
    post and not the other... now, get this, both MSGIDs, even though
    they are the same, are in the dupe database but with different
    hashes...

    I fail to see what's your point here. They weren't dupes according to
    hpt, so the different crc/hash's were stored.

    the point is that they ARE dupes and should have been detected as
    dupes... someone's formula is incorrect... it reminds me of a mailer/tosser combo that takes the header plus the first 40 bytes
    to calculate their hash with... software developers need to know
    about this so they can take steps to put the data in the proper
    place so this particular software will see the data and catch dupes
    as needed... this means to place the MSGID at the top or just after
    the AREA tag... but see? this is the type of knowledge that is
    gained after years of working with FTN software and being aware of
    how things were done in the past... not assuming that everything is
    done the same everywhere... especially since many developers are
    now gone and their knowlegede is no longer first or second hand...
    even with new developers having taken over older software and the
    details of the operation not having been transmitted along with the
    actual code base... the new maintainers have to figure it out for themselves or be told by someone that knows these details...
    reading the code is one thing... understanding it and why it was
    done a certain way is another...

    )\/(ark

    Always Mount a Scratch Monkey
    Do you manage your own servers? If you are not running an IDS/IPS
    yer doin' it wrong...
    ... I'm not an asshole. I'm a hemorrhoid. I irritate assholes. ---
    - Origin: (1:3634/12.73)
    @EEN-BY: 1/10 146 11/0 1 25/8 57/0 1 73/0 1 100/200 111/1 120/229
    @EEN-BY: 120/301 302 323 502 544 545 546 640 123/57 500 130/505 138/146 @EEN-BY: 140/1 153/250 7715 154/10 199/0 1 200/1 203/0 220/10 221/0 @EEN-BY: 226/301 227/51 230/0 240/5832 249/303 261/38 266/512 275/100 @EEN-BY: 280/464 5003 288/34 317/2 342/17 77 352/0 3 393/68 423/120 @EEN-BY: 712/848 770/0 1 100 340 772/0 1 100 210 777/7 863/100 101 @EEN-BY: 902/508 2215/15 300 1701 2320/100 3828/7 4975/301
    @ATH: 3634/12 123/500 261/38 393/68 770/1 280/464 227/51 120/544
    @ATH: 140/1 3828/7


    Regards,

    Roger
    --- timEd/386 1.10.y2k+ W10 (1607)
    * Origin: NCS BBS - Houma, LoUiSiAna - (1:3828/7)
  • From mark lewis@1:3634/12.73 to Roger Nelson on Tue Nov 1 20:14:52 2016

    01 Nov 16 15:21, you wrote to me:

    @ATH: 3634/12 123/500 261/38 393/68 770/1 280/464 227/51 120/544
    @ATH: 140/1 3828/7

    yes, i suspect it is one (at least) mystic system in the above path... i know there is one... i'm not sure if there is more than one right now... i hate feeling like triple warmed over death :(

    i just saw the tag line golded+ chose for this post :lololol:

    )\/(ark

    Always Mount a Scratch Monkey
    Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
    ... Any chance these could be hallucinations?
    ---
    * Origin: (1:3634/12.73)
  • From Roger Nelson@1:3828/7 to mark lewis on Wed Nov 2 06:36:48 2016
    On Tue Nov-01-2016 20:14, mark lewis (1:3634/12.73) wrote to Roger Nelson:

    01 Nov 16 15:21, you wrote to me:

    @ATH: 3634/12 123/500 261/38 393/68 770/1 280/464 227/51 120/544
    @ATH: 140/1 3828/7

    The problem seems to be around 3:770/1, allthough I have a high regard for Paul
    Hayton, anyone can overlook something, so it may not be him.
    .
    yes, i suspect it is one (at least) mystic system in the above
    path... i know there is one... i'm not sure if there is more than
    one right now... i hate feeling like triple warmed over death :(

    i just saw the tag line golded+ chose for this post :lololol:

    Yes. Mine are routine. (-:

    ... Any chance these could be hallucinations?


    Regards,

    Roger
    --- timEd/386 1.10.y2k+ W10 (1607)
    * Origin: NCS BBS - Houma, LoUiSiAna - (1:3828/7)
  • From mark lewis@1:3634/12.73 to Roger Nelson on Wed Nov 2 09:40:24 2016

    02 Nov 16 06:36, you wrote to me:

    On Tue Nov-01-2016 20:14, mark lewis (1:3634/12.73) wrote to Roger Nelson:

    01 Nov 16 15:21, you wrote to me:

    @ATH: 3634/12 123/500 261/38 393/68 770/1 280/464 227/51 120/544
    @ATH: 140/1 3828/7

    The problem seems to be around 3:770/1, allthough I have a high regard for Paul Hayton, anyone can overlook something, so it may not be him. .

    3:770/1 runs fastecho... i'm thinking it is another system in the path and unfortunately the problem won't be fixed for another little while...

    )\/(ark

    Always Mount a Scratch Monkey
    Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
    ... Ancient Chinese Curse: May all your wishes be granted.
    ---
    * Origin: (1:3634/12.73)
  • From Roger Nelson@1:3828/7 to mark lewis on Thu Nov 3 04:50:21 2016
    On Wed Nov-02-2016 09:40, mark lewis (1:3634/12.73) wrote to Roger Nelson:

    02 Nov 16 06:36, you wrote to me:

    On Tue Nov-01-2016 20:14, mark lewis (1:3634/12.73) wrote to
    Roger Nelson:

    01 Nov 16 15:21, you wrote to me:

    @ATH: 3634/12 123/500 261/38 393/68 770/1 280/464 227/51 120/544
    @ATH: 140/1 3828/7

    The problem seems to be around 3:770/1, allthough I have a high
    regard for Paul Hayton, anyone can overlook something, so it may
    not be him. .

    3:770/1 runs fastecho... i'm thinking it is another system in the
    path and unfortunately the problem won't be fixed for another
    little while...

    I hope I live to see that. (-:


    Regards,

    Roger
    --- timEd/386 1.10.y2k+ W10 (1607)
    * Origin: NCS BBS - Houma, LoUiSiAna - (1:3828/7)