Scribd
Upload
30 views11 pages

Hydranode Project - Edonkey2000 Protocol

This document describe the eDonkey200 (ED2K) P2P protocol and eMule Extended Protocol (EMEP)

Uploaded by

DeadMike
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
30 views11 pages

Hydranode Project - Edonkey2000 Protocol

This document describe the eDonkey200 (ED2K) P2P protocol and eMule Extended Protocol (EMEP)

Uploaded by

DeadMike
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
2ov7e 8 Hysranode Project Download Hydranode Hydranode Project Developer’ s Diary Home | Downloads | Documentation | Blog | DevCenter | FAQ ow Main Page | Namespace List | Class Hierarchy | Class List | File List | Namespace Members | Class Members | File Members | Related Pages | Search for eDonkey2000 Protocol Abstract This document describes the eDonkey2000 (&D2K) P2P protocol and eMule Extended Protocol (EMEP). Table of Contents Introduction * Definitions 2, Header specification 3, Client-Server communication © 3.1 Logging into a server © 3.2 Updating server list and i 3.2 Publishing Shar * 3.3 Performing a search mation d Files 4. Client <-> Client communic © 4.1 get-to-know-you chit-chat © 4.2 Regesting a file © 4.3 Uploading file data 5 LowID Clients © 5.1 Connecting to LowID clients Source Exchange Secure Identification Miscellaneous TCP packets UDP p: ets 1, Introduction ED2K protocol is used for communication between compatible clients to form a server-based peer-to-peer file-sharing network. EMEP is only used between eMule (and compatible) clients, but not by official eDonkey2000 clients, Few words on the packet format strings used here: 08 unsigned 8-bit integer ul6 unsigned 16-bit integer u32 unsigned 32-bit integer hash 16-bytes mé4 checksum bipimyérancde.ccmidocsled2vec2igto ppogsry wm zov7e 8 Hysranode Project string 16-bit integer (specifying string length) followed by the string Taglist 32-bit integer (specifying number of tags) followed by the tags Definitions * part is a region of a file of (up to) 9500kb in length, for which there is a corresponding MD4 PartHash. + chunk is a region of a file of (up to) 180kb in length, which can be requested from a remote client. * block is a region of a chunk which is sent as single packet during data transfer between clients. Chunks are split into block packets for transmitting. See also: eDonkey2000 Tag System Overview Header specification protocol | packet length | packet data (length bytes) Each eDonkey2000 packet begins with protocol opcode, followed by 32-bit unsigned integer indicating the packet length. Currently, three protocol opcodes are in use: PR_ED2K PR PRIZLTB Oxe3, //!< Standard, historical eDonkey2000 protocol Oxe5, //!< ellule extended protocol Oxdd//!< Packet payload is compres ed with gzip Packed data always begins with 8-bit opcode, which indicates the packet content. When sending packets using PRZLIB protocol, the packet data (excluding the opcode) is compressed. Client-Server communication 3.1 Logging into a server In order to connect to an eDonkey2000 Server, Client sends OP_LOGINREQUEST to server. OGINREQUEST = Ox01, //16 hash(u32>ipCul6>port tags The packet may contain the following tags: 1 NICK = 0x01, //1€ 0x30 ct_porr ox0F, //!¢ port TIMULEVERSION = Oxf, //1< €u82>ver cToFLAGS = 0x20, //1K flags Flags is a bitfield containing the following information: FLLZLIB = 0x01, //1< zlib compression support FLLIPINLOGIN 0x02, //1€ Client sends its own ip during login PLIAUK! oxod, //1¢ 997 FLINEW 0x08, //1¢ support for new-styled eMule tags FLIUNTCOD: = 0x10 //1 support for unicode On a well-formed OP_LOGINREQUEST packet, the server first attempts to connect to the sending client to its reported listening port as a normal client. If the bipimyérancde.ccmidocsled2vec2igto ppogsry ant zov7e 8 Hysranode Project connection attempt succeeds, and the server receives @ wellformed OP_HELLOANSWER packet from the socket, the server assigns the connecting client High ID. If the connection fails for whatever reason, the client is assigned Low ID. See also: 5 LowID Clients The following packets are sent by the server to inform the client that the connection has been established, as well as to provide up2date information about the server OP_SERVERMESSAGE = 0x38, //!< ¢ul6>Len¢Len>message OPLSERVERSTATUS = 0x34, //1< count | {¢u32>ipCul6>port} *eount) Oxd1, //!< Chash>hash(u32>ipCul6>portTagList> tags oP_SERVERLIST OP_SERVERIDENT The latter contains the following tags: RVERNAME -RVERDES 0x01, //1< neme Oxob //!< dese 3.2 Publishing Shared Files After establishing connection with the server, the client must publish it’s shared files using OP_OFFERFILES packet. The packet should be compressed if the server supports it to save bandwidth. 7/1 6u32>count [Ccount># (Chash> Ti lehash(u32>ipCul6>port tags} ) OP _OFFERFT = 0x15, The packet may contain the following tags: CT_PILENAME = 0x01, //'6 name CT_FILESIZE 0x02, JIN size ct FILETYPE = 0x03, JIG type Note: File type is sent in ed2k network as string. The following strings are recognized: #define “audio” //L6 mp3/oge/wma ete define “Video” J/'S avi/mpg/mpeg/wmy ete #define FT “Image” —//1< png/jpg/gif/tift ote Hdefine FT_ED2K_DOCUMENT “Doc” JAS txt/doc/rtt ete define PTLED2K PROGRAM “Pro” //\K exe/din/euesiso ete bipimyérancde.ccmidocsled2vec2igto ppogsry ant zov7e 8 Hysranode Project This packet should be sent on the following occasions * With full shared files list when connecting to server + With single file whenever new shared file is added + As cupty packet, as server keep-alive packet at regular intervals Additionally, it is possible to indicate whether the file being shared is partial or complete using two special ip/port values: CONPLETE_ID = Oxfefefefe, //1< File is complete ~ send this as ID F FLLCOMPLETE_PORT = Oxfef //1 File is complete ~ send this as port FL_PARTIAL_ID oxtbfbfetb, //1< File is partial - send this as ID FLLPARTIAL_PORT = Oxfbfb J/S File is partial ~ send this as port Note: Only use these ID's on newer eservers. eMule us whether the server supports these id's too. 3.3 Performing a search Searching in eDonkey2000 network is server-based. Client sends SEARCIIR packet to server, which in turn returns one (or more) SEARCHRESULT packets. This is done over TCP, and with currently connected server only. However, optionally, the client may also send the same search request packet over UDP sockets to all known servers using GLOBSEARCI opcode, wh packets. format specifi- cation is beyond the scope of this document. Refer to ED2KPacket::Search implementation for more information (packets. cpp) op_SEARCIL = oxi6, JING Ssearchexpr> 7/ 6u32>count [# (hash(u32> iéCul6>porttags) } OP_SEARCHRESULT = 0x33, SearchResult may contain the following tags: T_BILENAME 0x01, JG Sstring>name CT_PILESize 0x02, NG Sa82dsiz0 eT FILETYPE 0x03, JEM type cT_souRcES 0x15, “1 Sa82>numsre CT_COMPLSRC = 0330, 7/1 u32>numcomp | sre 4. Client <-> Client communication 4.1 get-to-know-you chit-chat In order to initialize a connection to a remote client, first the client sends Hello packet. oP_HELLO = 0x01, //1¢ Chash>hash(u32>ipCul6>porttags Hello packet may contain the following tags: = OxO1, //1 nick JEM Sul@port YING Su82>ver //16 modstring 1/1 Sul6>kadudppor ted2kudpport (116 Features bitsot crepe CTLMISCPEATUR: identical to OP_HELLO, except that HelloAnswer also includes 8-bit hash length before hash. The value of the field must be OxOf. CT_MULEVERSION contains a 32-bit value with the following bits (and meanings) bipimyérancde.ccmidocsled2vec2igto ppogsry zlib-support to detect ch in turn reply (over UDP) with GLOBSEARCIIRES The correct answer to OP_HELLO is OP_HELLOANSWER. The HelloAnswer packet format is ant zov7e 8 Hysranode Project 1 2 3 4 5 [bytes] 1111111122222933333334445555555 [bits] 09000000000000001010101100000000 eMule 42g version info Build version (unused by ellule) Update version 6 a= 0, g = 4) Minor version 42 Major version (unused by elfule) Compatible Client ID (CS_BMULE = 0x00) If both of the clients are eMule-compatible, they also exchange MuleInfo packets. oP_MULETNFO OP _MULBINFOANSWER x01, //1< el ientverprotvertags x02 //1< el ientverprot ver tags, Note: Both of these packets are sent using PRI MULE. Tags contained in Mulelnfo packets: CT_COMPRESSION //\ 482 compression version ct_eppport //1 u32 udp port cTLeDPVER 7/1 u32 udp protocol version CT_SOURCEEXCIL //\K 482 source exchange version cr 7/16 W2 comment version cr J/\K 482 extended request version CT _COMPATCL.TENT //'C uB2 compatible client 1D CTLFEATURES //1& u32 supported features bitset CT_MODVERSTON J/K modversion (nay also be int) T_MODPLUS| //'G wh? (Source: ellule> Forums ... ) CTLL2HAC //\ wh? (Source: ellule~ Forums...) Feature set bitfield is 32-bit value with following bits and meanings: 12345678123456781294567812345678 00000100000100110011001000011110 eMule 43b 00110100000100110011901000011110 ellule 446 11123333444455556066777788889abe fllottol Preview Itt Mul tipacket No “view shared files’ supported Peercache Comments Extended requests Source exchange — Secure ident ~ Data compression version — UDP version Un AICH version (0 ~ not supported) 4.2 Reqesting a file After completing the handshaking described in section 4.1, the client may now request a file. This is done using ReaFile packet. oP_REGFTL = 0x58, //1¢ hesh Note: eMule ExtReqV1 adds countbitarray partmap eMule ExtReqV2 adds completesources ReqFile is replied by OP_FILENAME and OP_FILEDESC = 0x89, //1< Shash>hash6u32>Lenlen>name = Ox6L, //1< rating1encomment, bipimyérancde.ccmidocsled2vec2igto ppogsry sit zov7e 8 Hysranode Project After receiving the above packets, the downloading client sends SETREQFILEID to bind the reqested file to the hash, This means the client is now bound to the requested hash, until it receives it, FileDese packet is sent using PR_EMULE, and does NOT contain the file hash the description belongs to, however, since it is always sent along with OP_FILENAME, in response to OP_REQFILE packet, client can thus assume the FileDese packet belongs to the most recently requested file. Note: Actually, eMule extended protocol allows changing the reqested file while waiting in the queue by resending OP_SETREQFILETD at any time OP_SETREQPILEID = Ox4f, //1¢ hash The expected response to this packet is OP_FILESTATUS OP_REQFILE_STATUS = 0x50, //!< hashcountpartmap Tf at any point during the above packets the uploading client realizes it doesn’ t share the file requested, it may send OP_REQFILENOFILE. File status packet contains a part map of the available parts. This is defined as a bitfield, one bit per each ED2K part (defined by ED2K_PARTSIZE). Extra bits needed to fill up the byte are padded with zeros. The count prior to the bitset indicates the number of total parts, and thus also the number of total bits in the bitset (note: NOT the nunber of bytes following!). Also note that sending partmap is completely optional, and should only be sent if the file is indeed partial ~ if the sender has the entire file, the partmap shall be omitted. OP_REQFILE_NOFILE = Ox48, //1< hash If the downloading client needs a hashset of the file, it may ask the uploader for it using OP_REQHASHSET. The expected response is OP_HASHSET. oP_REQHISHSET = Ox51, //1< hash oP UASUSET = 0x52) //1< hashent [ent&parthash] After that, the requesting client sends OP_STARTUPLOADREQ, which must be replied either by OP_ACCEPTUPLOADREQ (if we can start uploading right away), or ‘OP_QUEUERANKING, in case the client has been inserted into our upload queue. OP_STARTUPLOADREG = Ox54, //1< may contain hash (emule) op ‘UPLOADREQ = 0x55, //1< Empty RANKING = OxBe, //1€ queueranking There is another queue ranking packet used by eMule ~ denoted by opcode ‘OP_MULEQUEUERANK. op_Mw FRANK = 0x60 //1€ quouerank empty This packet contains 16-bit value as queue rank, and 10 empty bytes, such as the full packet size must be 12 bytes. This size requirement is enforced by eMule packet parser, and the sender will be disconnected if those extra bytes are not sent. Why are there those 10 extra empty bytes, and why is it enforced so strongly remains unclear. This packet is also sent using PR_EMULE protocol opcode. Note that if you'r thinking to skip over this packet and not use it, you'r so out of luck, since eMule considers clients who send OP_QUEUERANKING simply QueueFull (altough it seems to parse the packet). So if you want eMule compatibility, you need to use this half-empty packet. ploading file data hipimyrancde.ccmidocsled2vec2igto ppogsry ant zov7e 8 Hysranode Project After receiving OP_ACCEPTUPLOADREQ, the remote client will proceed to request chunks from us. In eDonkey2000 network, a chunk size is 180k (notice the difference from partsize). The chunks are requested using OP_REQCHUNKS packet: op_REQCHUNKS = 0x47, //!< Shash>hash[3+*bogin] [3#end] The packet contains three begin offsets and three end offsets of the requested chunks. £b2K data ranges logic defines that the end offset is exclusive, thus range (0, 0) is considered as invalidrange. If the client wishes less than three chunks, it may set the remaining ranges to (0, 0) to indicate that. Note: This is slightly different from HydraNode Range API, where end offset is considered inclusive. Implementors should be warned about different behaviours of different clients when it comes to chunk requests. Namely, eMule (and compatible) clients use “rotational chunkrequest” scheme, where each REQCHUNKS packet contains one new chunk and two older chunks; for example: Packetl: 0,101, 101,.201, 201..301 // initial req Packet2: 101..201, 201..301, 301..401 // sent after completing 0.101 Packet3: 201.301, 301,.401, 401,.501 // sent after completing 101.201 MLDonkeys, and some other clients, however, only send single requests PacketI: 0..101, 101.201, 201..301 // initial req Packet2: 301..401, 0.0, 0.0 // after completing 0.. 101 Packet3: 401..501, 0..0, 0.0 // after completing 101. . 201 This can lead to duplicate data being sent by mldonkeys, if a rotational chunkrequest scheme is used when communicating with them, For that reason, HydraNode uses non-rotational request scheme with all clients, since all clients handle it properly. After receiving OP_REQCHUNKS, the uploading clients starts sending data. The requested chunks are split into (up to) 10kb blocks, each transmitted as a separate packet, using OP_SENDINGCHUNK opcode. Optionally, clients supporting eMule extended protocol may use OP_PACKEDCHUNK packet (which is sent using PR_ENULE). In compressed packet, the data part of the packet is compressed using zlib (but not the packet header ~ this is different from OfferFiles packet, where everything after the opcade is compressed). Also, in case of a compressed packet, the end offset bytes in the packet header indicate the length of the compressed data, not the end offset of the uncompressed data. Note that compression is not required - if data compression results in larger amount of data, data should be sent uncompressed instead, using OP_SENDINGCHUNK packet. op INDINGCITUNK \CKEDCHUNK x46, //< Chash>hash(u32>boginendCdata> x40, //1< Chash>hash>beginsu32>LenClendata When the receiver decides it does not want any more data from us, it will send OP_CANCELTRANSFER packet (empty payload) to the uploader. OP_CANCELTRANSPER = 0x56 //!< empty In case of compressed data transfer, the entire requested chunk is first compressed, and then transmitted in 10kb chunks over time. 5 LowID Clients LowID client is defined in ed2k network as a client who cannot receive external connection, The client can only make external connections itself. The only way to connect to a client is through a server callback, since the client is connected to bitp:tyranade com tocsled2ied2krao ppogsry mt zov7e 8 Hysranode Project a server, It is only possible to contact LowID clients that are on same server as the connecter is. LowlD client's are identifier in the network using client ID less than Ox00ffffff. This is also the reason why everywhere where there should be client IP we say it is ID. For clients with ID > OxO0ffffff, the ID indicates the IP of the remote client (in network byte order), for lower TD's, it indicates the client callback ID, as assigned by the server. 5.1 Connecting to LowID clients Connecting to LowID client is done by sending OP_REQCALLBACK to currently connected server, including the ID of the remote client we wish to connect. OP_REQCALLBACK = Oxle, — //1< id If the client with the specified ID is connected to the server, the server sends ‘OP_CBREQUESTED packet to the client, after which the client will attempt to contact the requester. OP_CPREQUESTED = 0x35, //!1< $uB2>ipCul6>tepport If the client in question is not connected to this server, the server will respond with OP_CALLBACKFATL OP_CALLBACKPAIL = 0x38 //1< empty 6 Source Exchange Extended protocol feature added by eMule, Source Exchange allows clients to share their known sources for files. REQSOUR TANSWERSOURCES: Ox81, —//1€ hash 0x82, Note: SreExchv2 adds 16-byte user-hash to each source. SrcExchy3 sends ID's in byteswapped format (also called “Iybrid” format in eMule code) so HighID clients with **,*.0 won't be falsely given Lowld. Normally, this packet is sent using PREMULE, but if possible, it should be compressed and sent using PR_ZLIB. Note 2K netiquette specifies that you should not request mosources from clients at intervals exceeding these: + No more than once per 40 minutes per client in case of rare files (Less ‘than 10 sources) + No more than once per 2 hours per client in case of common files 7 Secure Identification Secure Identification provides means of determining the client’s identity in a secure manner, and avoid the former problems regarding hash/credit stealers. In the protocol, three packets implement SecIdent: OP_SECIDENTSTATE = 0x87, //1¢ SuB>statecua2>chal lenge Request signature and/or publickey from remote client oP_PUBLICKI = 0x85, //!< ¢u8>lendlen>pubkey bipimyérancde.ccmidocsled2Wec2igxcto ppogsry ant rors ydranode Project, Includes the senders public key oP. TURE = Ox86 //1¢ CuB>lenClen>signature Includes the senders signature The signature message contents are as follows: ®pubkeychal lenge Note: Secldentv2 adds u32 ip and u8 iptype The pubkey is the signature requesters public key. The message is signed with the senders own private key. The receiver can then verify the message with the senders public key. Sample implementation of this scheme, using CryptoPP library, CreditsDb: :createCryptKey CreditsDb::loadCryptKey CreditsDb: :createSignature Creditsbb: :verifySignature 8 Miscellaneous TCP packets OP_MESSAGE = Oxde, — //1X Len0x12, Client: 1 Not actually part of ed2k protocol, OP_PORTTEST can be used (by a website, for example) to verify correct firewall configuration, Note that OP_PORTTEST may be sent both via TCP and UDP. OP _CHANGELD = Oxdd — // 1K 01didu32>newid Sent when client changes it’s ip address, or in case of LowID clients, changes it’s ID (sent by server). Mules seem to only handle this packet, but not send it themselves. Note: The actual format of this packet has not yet been fully verified. 9 UDP packets The format of UDP packets header differs slightly from TCP packet headers, namely, the size field is omitted, and opcode follows the protocol code. 9.1 cMule extended protocol: UDP reasks As an eMule extended feature, UDP packets are used to “ping” sources every now and then to verify queue status, reask queue rankings and so on. In current implementation, eMule (and compatible) clients seem to drop clients from their queues if they haven’ t pinged at least once per hour, The following packets are currently in use, and are sent using protocol PR_EMULE: OP_RPASKPILEPING = 0x90, //1 hash By default, this packet only contains filehash the source was interested in. Note: UDPv3 adds 2-bytes complete-source-count. UbPv4 adds availability partmap, as in ReqFile packet. bipimyérancde.ccmidocsled2vec2igto ppogsry ont zov7e 8 Hysranode Project When UDPv4 is used, the partmap is placed before the complete source count, oP_REASKACK = 0x91, //1< Global Server communication, UDP is used for global searches, as well as global source queries. The following packets are being used, and sent using protocol PR_ED2K: OP_GLORGETSOURCES = O98, //1< ent#[hash] All servers support this packet, but if the ServerUDPFlags includes FL_GETSOURCES, the server allows requesting sources for multiple files at same time, When sending more than one hash, care must be taken to limit the packet size to 512 bytes, anything above that will be ignored by server. ope ETSOURCES2 = Ox94, //1< ent*[hashu32>size] Modified version of the above packet, this also includes file-size in the request. This can only be used with servers, which support FL_GETSOURCES2 flag. Note: You shouldn’t query any server twice for sources during 20-minute interval, or you might get banned. GLOBPOUNDSOLRCES = Ox9b, //!< Chash>hashent#[i dport] The expected response to OP_GLOBGETSOURCES or GLOBGETSOURCES2 packets, this packet contains the list of sources the server knows, TATREQ = 0x96, //!< chal lenge Used for “pinging” global servers, this packet requests the remote server to return info about it’s current state. Servers always respond to this packet (if alive), so this can be used to determine whether the server is alive or not. As with GlobGetSources, no server should be queried more often than once every 20 ninutes. The first two bytes of the challenge (in network byte order) are alw. 0x55aa, followed by random 16bit integer. set to oP_GLOBSTATRES = 0x97 bipimyérancde.ccmidocsled2Wec2igxcto ppogsry sont zov7e 8 Hysranode Project The expected response to OP_GLOBSTATREQ, this packet contains bunch of information about the server. Only the first three fields (challenge, users, files) are alwe present, the remainder of fields are optional (only implemented by newer server versions, and can possibly be omitted even by newer servers). The challenge is equal to the challenge sent in OP_GLOBSTATREQ packet. Ss Copyright © 2004-2006 Alo Sarv bipimyrancde.ccmidocsled2vec2ixto ppogsry aw

You might also like