星际联盟提供对Filecoin Spec 的全文翻译,便于Filecoin项目广大中国参与者理解Filecoin的深层原理。文章将定期更新章节,请持续关注"IPFS星际联盟"&"星际联盟Filecoin"公众号。
2.2.3.5 交替合并流程 - 单轮往返
数据传输 - 单轮往返合并流程(在新标签页中打开)
- 当请求者想要从另一方接收数据时,它会发起合并(Pull) 传输
- 请求者的数据传输模块(DTM) 调度安排数据传输
- 请求者通过数据传输请求向响应者发出Graphsync请求
- 响应者接收到Graphsync请求,并将数据传输请求转发到数据传输模块
- 请求者的数据传输模块会将合并请求连同数据传输凭证一起发送到响应者
- 响应者的数据传输模块通过使用PullValidator(合并验证器) 验证数据传输请求,该验证器作为附属品,由响应者提供
- 响应者的数据传输模块安排调度传输
- 响应者发送Graphsync响应,并附带一个数据传输接受的响应:piggypacked
- 请求者接收数据并可以产生进度指示
- 请求者完成接收数据,并通知所有listeners(监听者)
2.2.3.6 协议
数据传输可以通过数据传输协议(Libp2p协议类型)在网络上协商
代码合并请求需要响应。 在知道请求已被接受之前,请求者不会启动传输。
响应者也应发送对推送请求的响应,以便请求者可以释放资源(如果未接受)。 但是如果响应者接受了请求,便可以立即启动传输。
使用数据传输协议作为独立的libp2p通讯机制并不是硬性要求-只要双方都实现了可以与对方通信的数据传输子系统,任何传输机制(包括离线机制)都是可以接受的。
2.2.3.7 数据结构
import ipld "github.com/filecoin-project/specs/libraries/ipld"
import libp2p "github.com/filecoin-project/specs/libraries/libp2p"
import cid "github.com/ipfs/go-cid"
import piece "github.com/filecoin-project/specs/systems/filecoin_files/piece"
import peer "github.com/libp2p/go-libp2p-core/peer"
type StorageDeal struct {}
type RetrievalDeal struct {}
// A DataTransferVoucher is used to validate
// a data transfer request against the underlying storage or retrieval deal
// that precipitated it
type DataTransferVoucher union {
StorageDealVoucher
RetrievalDealVoucher
}
type StorageDealVoucher struct {
deal StorageDeal
}
type RetrievalDealVoucher struct {
deal RetrievalDeal
}
type Ongoing struct {}
type Paused struct {}
type Completed struct {}
type Failed struct {}
type ChannelNotFoundError struct {}
type DataTransferStatus union {
Ongoing
Paused
Completed
Failed
ChannelNotFoundError
}
type TransferID UInt
type ChannelID struct {
to peer.ID
id TransferID
}
// All immutable data for a channel
type DataTransferChannel struct {
// an identifier for this channel shared by request and responder, set by requestor through protocol
transferID TransferID
// base CID for the piece being transferred
PieceRef cid.Cid
// portion of Piece to return, specified by an IPLD selector
Selector ipld.Selector
// used to verify this channel
voucher DataTransferVoucher
// the party that is sending the data (not who initiated the request)
sender peer.ID
// the party that is receiving the data (not who initiated the request)
recipient peer.ID
// expected amount of data to be transferred
totalSize UVarint
}
// DataTransferState is immutable channel data plus mutable state
type DataTransferState struct @(mutable) {
DataTransferChannel
// total bytes sent from this node (0 if receiver)
sent UVarint
// total bytes received by this node (0 if sender)
received UVarint
}
type Open struct {
Initiator peer.ID
}
type SendData struct {
BytesToSend UInt
}
type Progress struct {
BytesSent UInt
}
type Pause struct {
Initiator peer.ID
}
type Error struct {
ErrorMsg string
}
type Complete struct {}
type DataTransferEvent union {
Open
SendData
Progress
Pause
Error
Complete
}
type DataTransferSubscriber struct {
OnEvent(event DataTransferEvent, channelState DataTransferState)
}
// RequestValidator is an interface implemented by the client of the data transfer module to validate requests
type RequestValidator struct {
ValidatePush(
sender peer.ID
voucher DataTransferVoucher
PieceRef cid.Cid
Selector ipld.Selector
)
ValidatePull(
receiver peer.ID
voucher DataTransferVoucher
PieceRef cid.Cid
Selector ipld.Selector
)
ValidateIntermediate(
otherPeer peer.ID
voucher DataTransferVoucher
PieceRef cid.Cid
Selector ipld.Selector
)
}
type DataTransferSubsystem struct @(mutable) {
host libp2p.Node
dataTransfers {ChannelID: DataTransferState}
requestValidator RequestValidator
pieceStore piece.PieceStore
// open a data transfer that will send data to the recipient peer and
// open a data transfer that will send data to the recipient peer and
// transfer parts of the piece that match the selector
OpenPushDataChannel(
to peer.ID
voucher DataTransferVoucher
PieceRef cid.Cid
Selector ipld.Selector
) ChannelID
// open a data transfer that will request data from the sending peer and
// transfer parts of the piece that match the selector
OpenPullDataChannel(
to peer.ID
voucher DataTransferVoucher
PieceRef cid.Cid
Selector ipld.Selector
) ChannelID
// close an open channel (effectively a cancel)
CloseDataTransferChannel(x ChannelID)
// get status of a transfer
TransferChannelStatus(x ChannelID) DataTransferStatus
// pause an ongoing channel
PauseChannel(x ChannelID)
// resume an ongoing channel
ResumeChannel(x ChannelID)
// send an additional voucher for an in progress request
SendIntermediateVoucher(x ChannelID, voucher DataTransferVoucher)
// get notified when certain types of events happen
SubscribeToEvents(subscriber DataTransferSubscriber)
// get all in progress transfers
InProgressChannels() {ChannelID: DataTransferState}
}
2.2.4 数据格式和序列化
Filecoin旨在使用尽量少的数据格式,采用规范化的序列化规则,以更简单更好的方式提高协议安全性,并实现Filecoin协议实现之间的互操作性。
更多关于设计方面的考虑参见 here for CBOR-usage 和 here for int types in Filecoin.
2.2.4.0.0.1 数据格式
Filecoin内存中的数据类型大多很简单。
实现应支持两种整数类型:Int(指native 64位整数)和BigInt(指任意长度),并避免处理浮点数以最大程度地减少跨编程语言和实现的互操作性问题。
更多内容参见Filecoin协议中data formats as part of randomness generation(数据格式作为随机生成的一部分)
2.2.4.0.0.1 序列化
Filecoin中的数据序列化(Serialization
) 需要确保一致的格式,以便对in-memory数据进行序列化以进行 in-flight和in-storage传输。 序列化对于Filecoin协议实现之间的协议安全性和互操作性至关重要,并可以跨Filecoin节点进行一致的状态更新。
Filecoin中的所有数据结构都是CBOR元组(CBOR-tuple) 编码的。也就是说,Filecoin系统中使用的任何数据结构(本细则中的结构)都应该序列化为具有与数据结构字段想对应的项的CBOR-arrays,这些项按声明的顺序排列。
CBOR中的主要数据类型的编码结构参见这里
为了便于说明,内存中的映射将表示为键和值的CBOR-array,这些键和值按预先确定的顺序列出。对序列化格式的近期更新将涉及对字段进行适当的标记,以确保随着协议的发展出现适当的序列化/反序列化。