Upload priority proposal

User avatar
esher
Site User
Posts: 46
Joined: Sun Apr 08, 2012 9:42 pm

Upload priority proposal

Postby esher » Sat Jul 04, 2015 8:01 pm

the idea is to push the file less transferred x network
tracking requesters chunk/piece composition for a single file using their reasks to etablish if there is any change - all requesters from a ntwork for each file - (if they acquired some data or not - improvement or stall)
the file with the longer stall status will get the slot. completing a succesfull upload session is count as an improvement and the file will go bottom queue.
downloads/part files will be reverse, last part file involved in any improvement (successfully downloaded part counts) will be the one.
give 50% chance to be a part or a complete upload slot and keep chosing randomly the client.
in other words... the complete file with the slowest parts transfer among requesters will be the one (any successful upload from our side will put it 0.00),
for downloads: still track other requesters reask status changing (if we just had a part it's an improvemt too - 0.00) the file that had just an improvement will be the one.

the goal is to push the less transferred file according the real file speed not to count how many 5000queue client sources it has.

User avatar
OwenBurnett
Site User
Posts: 55
Joined: Fri Feb 27, 2015 9:25 pm

Re: Upload priority proposal

Postby OwenBurnett » Tue Jul 07, 2015 8:46 am

In 98% of cases the file speed is lower for the rare files, so those are the ones that should be pushed. But not just because of that, but to avoid their disappear ;)

If a file is requested by 100 clients, while another one is requested by 5, its obviously this last one that need some extra "tickets" to get a chance to get in the upload slots.
I won a chess game against David Xanatos :D

User avatar
esher
Site User
Posts: 46
Joined: Sun Apr 08, 2012 9:42 pm

Re: Upload priority proposal

Postby esher » Tue Jul 07, 2015 12:40 pm

iirc neomule worked pretty much as you just described, the idea was good but it poorly failed in some situation.
1. all files with one request were pushed massively. Did they really need that push? i tried to search from a freind home same files and not all of them needed a particular push.
2. whenever you had more requests for the same file, u could often see the requesters were all stuck at the same % with the same parts. this files had no chance to be pushed. the more requesters got stuck at that point the lower the fil prio went.

relying on complete sources count from source Exchange is failing because you know, fine, there are 9 sources + me, but they clearly made poor priority choices. and reverse when there were 2 sources + me... there could be a good sharer.

i think we need something highly dynamic and crossprotocl effective. edit: or non crossprotocol effective... who cares!? :P

User avatar
OwenBurnett
Site User
Posts: 55
Joined: Fri Feb 27, 2015 9:25 pm

Re: Upload priority proposal

Postby OwenBurnett » Wed Jul 08, 2015 1:11 am

The point is not just the speed (where the system works in most of cases though, because we have to assume that the majority of the time an higher number of sources has a potential higher speed, exception are really really rare), but the lifetime of the file. With 2 sources only, it may be avaible only at certain times because both clients might be offline for days, or might disappear completely... that's why rare files really need a push, to increase the number of sources (with full file) avaible.

Its pratically impossible to trace "filespeed" accurately enough to make a priority system based on that...
I won a chess game against David Xanatos :D

User avatar
esher
Site User
Posts: 46
Joined: Sun Apr 08, 2012 9:42 pm

Re: Upload priority proposal

Postby esher » Thu Jul 09, 2015 1:01 pm

btw the current random queue with default priority is basically a "follow the majority" mode . it's bad, even from your point of view because file rarity is brutally inrelevant.

User avatar
OwenBurnett
Site User
Posts: 55
Joined: Fri Feb 27, 2015 9:25 pm

Re: Upload priority proposal

Postby OwenBurnett » Thu Jul 09, 2015 1:20 pm

esher wrote:btw the current random queue with default priority is basically a "follow the majority" mode . it's bad, even from your point of view because file rarity is brutally inrelevant.


What do you mean with "follow the majorty" ? That file with more requests usually get more upload slot ? Well, its kinda obvious... to avoid that we should increase the "tickets" for the rare files. But on the other side, we should not make a brutal difference otherwise we won't upload the popular files in that situation.
I won a chess game against David Xanatos :D

User avatar
esher
Site User
Posts: 46
Joined: Sun Apr 08, 2012 9:42 pm

Re: Upload priority proposal

Postby esher » Thu Jul 09, 2015 3:00 pm

OwenBurnett wrote:Its pratically impossible to trace "filespeed" accurately enough to make a priority system based on that...


bummer.

OwenBurnett wrote:If a file is requested by 100 clients, while another one is requested by 5, its obviously this last one that need some extra "tickets" to get a chance to get in the upload slots.


obvious.

OwenBurnett wrote: But on the other side, we should not make a brutal difference otherwise we won't upload the popular files in that situation.


fair enough.

OwenBurnett wrote:file with more requests usually get more upload slot ? Well, its kinda obvious...


obvious, if there is just 1 complete source, me.

User avatar
OwenBurnett
Site User
Posts: 55
Joined: Fri Feb 27, 2015 9:25 pm

Re: Upload priority proposal

Postby OwenBurnett » Fri Jul 10, 2015 2:22 am

esher wrote:obvious, if there is just 1 complete source, me.


I understand your point, but we cannot force the other clients to upload more, expecially more than you :D

(or to manage the bandwitch in a smarter way)
I won a chess game against David Xanatos :D

User avatar
esher
Site User
Posts: 46
Joined: Sun Apr 08, 2012 9:42 pm

Re: Upload priority proposal

Postby esher » Fri Jul 10, 2015 11:51 am

you maybe misunderstood last sentence. im not sure...
i meant as long my nl is the ONLY complete source, the "tickets" won't decrease even in case of 5k requests.

User avatar
OwenBurnett
Site User
Posts: 55
Joined: Fri Feb 27, 2015 9:25 pm

Re: Upload priority proposal

Postby OwenBurnett » Fri Jul 10, 2015 12:40 pm

esher wrote:you maybe misunderstood last sentence. im not sure...
i meant as long my nl is the ONLY complete source, the "tickets" won't decrease even in case of 5k requests.


mhhhh... we should be using 1 ticket each client for the lower priority and maybe 20-30 ticket each client for the higher, I don't know exactly how it is right now, must wait for DavidX :P
I won a chess game against David Xanatos :D


Return to “Feature Requests”

Who is online

Users browsing this forum: Bing [Bot] and 1 guest

cron
Fatal: Not able to open ./cache/data_global.php