ndn libraries progress and plans · • complex namespace queries / pattern matching (i.e., regexp,...
TRANSCRIPT
Overview
• CommonClientLibraries(CCL)• PSync• CommonNameLibrary(CNL)• NDN-RTC• Quicksummaryofrecentresearchprogress
2
What are the Common Client Libraries (CCL)?
• EnableclientapplicationstouseNDNinC++,Python,JavaScript,Java,.NET• CommonAPIacrosslanguages:http://named-data.net/doc/ndn-ccl-api• Interest/Data,signatures,encryption,transports,apputilities,unittests,examples• Trackndn-cxxresearch(security,NAC,NDNprotocols,NFDinteraction)• Backwardscompatibility,platformflexibilityfordevelopmentstability• UsedinNDN-RTC,BMS,mHealth,neighborhoodnetwork,webpageapps,ICE-AR• Specializedlibraries:NDN-CPPLite(Arduino),Imp,Android,browserspeedups• Stats(total):10,771commits,277closedissues,79pullrequests,80forks
Example
face = Face("memoria.ndn.ucla.edu")
name = Name("/ndn/edu/ucla/remap/demo/ndn-js-test/hello.txt/%FDU%8D%9DM")
def onData(interest, data):
print(data.content.toRawStr())
face.expressInterest(name, onData)
CCL Features
• Certificatesigning/validating–RSA,ECDSA,HMAC• Configurablecertchainpolicies,regexnamematching• Flexiblepublic/privatekeydatabaseAPI• SignedInterests–verifywithsameAPIascerts• Name-baseaccesscontrol(AESencryption,RSAkeyprotection)• MemoryContentCache,SegmentFetcher• Optionalthread-safenetworkI/O• Configurablewireformat(seebelow)• ChronoSync,PSync(seebelow)• Unittests,exampleprograms
CCL wire format abstraction
• APIisnothard-wiredtoonewireformat• Enablebackwardscompatibilityifrunningwitholdforwarders
WireFormat.setDefaultWireFormat(Tlv0_1WireFormat.get())
• Canspecifyonadhocbasisifsendingtoavariousnetworksface.expressInterest(name, onData, Tlv0_1WireFormat.get())
• WasusedfortransitionfromCCN0.x• PlanstosupportotherICNwireformats
CCL – Next steps
• NDNwireformatv0.3(withbackwardscompatibility)• Typednamecomponents• Removed(most)Interestselectors• Interesthopcount• Interestdefaultstoexactname(optionalCanBePrefix)• ExtraapplicationparametersintheInterest• Explicitfieldsforsignedinterests(insteadofusingnamecomponents)
• Newwireformats• Supportnewnetworkautoconfigprotocols
What is PSync?
• DevelopedasimprovementtoChronoSync• UsedinNLSRtosyncroutesontheNDNtestbed• PartoftheCCL• InvertibleBloomfilterofasetofhashednames
• SendinterestwithmyIBF,receiveinterestswithothers’IBF• Stablestate:EveryonesendsthesameIBF–Interestaggregation,noData• Update:IreceiveadifferentIBFwithmissingnamesandprovideinreplyData• IBFefficientlyupdatesasetdifferenceof~275names
• Eventualconsistencyfrompairwiseupdates–broadcastnotneeded• Optiontosubscribetopartialnamespaceupdates
Example PSync app face = Face() def onNamesUpdate(names): print("Got names, starting with " + names[0].toUri()) updateSize = 80 pSync = FullPSync2017(updateSize, face, Name("/sync"), onNamesUpdate)pSync.publishName(Name(”/edu/ucla/jefft/paper.txt"))
PSync – Next steps
• ImplementinPython,JavaScript,Java(currentlyinC++)• UseasnativesyncfortheCommonNameLibrary(seebelow)• Stresstest“eventualconsistency”withoutbroadcast• SupportpartialPSync(waitingforusecase)• NDNProject:AQuickSummaryofRecentProgress
What is the Common Name Library (CNL)?
• Libraryenablingapplicationstoworkwithhierarchical,nameddatacollections.• Namespaceobject(rootandchildnodes)• ApplicationinteractswithaNamespacenode(attachhandlers,receivenotifications)
• Providesalightweightwaytointegratevarious:• Syncmechanisms(i.e.,PSync,vectorsync)• Dataaccesspatterns(i.e.,Consumer/ProducerAPI,fetchlatest),• Publishingmodels(i.e.,publish/subscribe,in-memorycontentcache),• Complexnamespacequeries/patternmatching(i.e.,regexp,wildcards),• Triggereddatageneration(supportingsecurity)
• CurrentlyusinginICE-ARmobileclientapplication(Nointerest-dataexchangeexposedtodevelopersofthatapp.)
• SegmentedcontentwithaMetapacketandversioning• Built-inencode/decode,encrypt/decrypt,sign/verifyaspartofthepipeline• NewnamesaddedtotheNamespacetreethroughPSync,appisnotified
CNL Motivation
• Providetoolsforworkingwithnamespacesastheyrepresentcollections,inaninformation-focusedratherthancommunication-orientedway• Assumeasynchronousnetworkoperationswillbeusedtosyncthenamespaceandconsume/publishobjectsinthecollection• Insulatenon-networkingdevelopersfromcommunicationdetails• MakeprogresstowardsNDNasamiddleware-replacementintermsofhigh-level,application-facingfeatures,buttrytostayasgeneralaspossible• Workwithaggregateapplication-specificobjects,not(segmented)blobsinpackets• Asaresult,supportnamespacesynchronizationthewaythatisconceived/describedatahigh-level,andpromoteitasanapplication-levelconcepttoexplore
12
Example segmented content consumer app face = Face("memoria.ndn.ucla.edu") page = Namespace("/ndn/edu/ucla/remap/demo/ndn-js-test/named-data.net/project/ndn-ar2011.html/%FDX%DC5B")page.setFace(face)
def onSegmentedObject(namespace): print("Got segmented object size " + str(namespace.obj.size()))
page.setHandler(SegmentedObjectHandler(onSegmentedObject)).objectNeeded()
CNL Handlers
• Supportextensibility• SetNamespaceforspecialfetching,publishing,objectrepresentation• UnifiedAPIfordeveloperstoo• https://github.com/named-data/PyCNL/blob/master/python/pycnl/segmented_object_handler.py
Unified publisher/consumer
• objectNeeded()–Fromapplication(producer)ornetwork(consumer)• Producer
• CNLreceivesInterest,addstoPIT,callsOnObjectNeeded(ifnotalreadyincache).• Handler’sOnObjectNeededanswersTrue.• CNLwaitsforapplicationtoproducedataasynchronously.• ApplicationcallssetObject().• CNLdoesserialize/encrypt/signandsatisfiesPIT.
• Consumer• ApplicationcallsOnObjectNeededforaNamespacenode.• (AllhandlersanswerFalse.)• CNLdoesFace.expressInterestandwaitsforData.• CNLreceivesData,doesverify/decrypt/deserializeandOnStateChanged(OBJECT_READY)
NAMEEXISTS
INTERESTEXPRESSED
DATARECEIVED DECRYPTING
OBJECTREADY
OBJECTTYPE/NAME
STRUCTUREKNOWN
Ifanaggregateobjectwith
versioning,forexample
OBJECTREADYBUTSTALE
Existinglisteners
VALIDATING(ALL)
VALIDATESUCCESS
(ANY)VALIDATEFAILURE
WAITINGFORDATA
(ANY)DECRYPTION
ERROR
ChilddatareceivedInteresttimeoutChildverificationerror
(ANY)INTERESTTIMEOUT
Signing/validationandencryption/decryptionmaybeperformedatboththepacketandobjectlevel,dependingontheobjecttype
(ANY)INTERESTNETWORKNACK
ToobjectNeeded
PRODUCINGOBJECT
AnOnObjectNeededanswerstrue
AllOnObjectNeededanswerfalse
ENCRYPTING
(ANY)ENCRYPTION
ERROR
SIGNING
(ANY)SIGNINGERROR
ReplytopendingincomingInterests
WheretostorechilddecryptedcontentBlobsbeforedeserializingthe
parentobject?
objectNeeded() SERIALIZING
setObject()
DESERIALIZING
NDN-CNL: Name node state diagram Integrating Interest/Data and Packet-/Prefix-level objects
CNL – Next steps
• High-performancepersistentstorage• PorttoJavaandJavaScript• Moreapplications
• Currentlyusedinaugmentedrealitymobileclientapplication
NDN Data Packets
/<base-prefix>
<timestamp>
<stream-name>
<seq #> _gop
_latest _live
_meta<segment #> _manifest _meta
_parity
frame payload manifest payload
Content-Type = 'ndnrtc'
Timestamp
Content-Size
Other
gop pos, gop #
capture timestamp
frame type
complete frame
parity size
generation delay
<segment #>
FEC payload
<seq #>
start end
/<stream-prefix>/<seq#> /<stream-prefix>/<seq#>
<version #>
0
1
/<stream-prefix>/<seq#>
/<stream-prefix>/_gop/<seq#>
<version #>
timestamp
publish rate
WxH
bitrate
description
What is NDN-RTC? • NDNC++video(HD)streaminglibraryformacOS,Ubuntu,Android• Sub-second(~150ms)latency• BasedonVP9videoencoder• Pipelining,retransmission,FEC• Unifiedconsumerforliveandstoredvideo• Recentupdates
• RDRprotocolforgettinglatestdata• RemovedependencyonWebRTC(thoughnoechocancellationpipelinefornow)• New“FastRepo”rocksdb-basedstorage• Movingtowardsgeneralizedapproachforobjects,metadata,etc.
github.com/remap/ndnrtc
Applications Current
• ICE-AR(ARbrowseraspartofIntel/NSFICN-WEN)• NDN-RTCstreamsphonePOVvideoforedgeprocessing(object,face,poserecognition)• processedinformationdeliveredbacktothephonetoenrichphone’senvironmentalunderstanding(deepcontext)
• TouchNDN(integrationwithDerivative’sTouchdesigner)• aimingtoreplaceNDIforlivevideoproduction• leverageNDNtoefficientlydisseminatelivevideooverL2(orL3)tomultiplenodesforsimultaneousprocessing&storage• nodesintegrate“historical”playbackfromrepodataseamlesslywithlivestreaming,forscrubbingreal-timestreams
• Assortedcommandlinetools
Previous
• ndncon/flume–pureP2Pconferencingapp• notuptospeedwithlatestlibrary
Future Plans
• Scalablevideocoding(VP9)• Region-of-Interest-basedfetching(360ovideousecase)• Volumetricvideostreaming• Congestioncontrol,whenappsneedit(basedonSchneider2016)
=>Lookingforappusersandcodebasecollaborators
Recent NDN Code Release Updates • NFDandndn-cxxversion0.6.5• ndn-toolsversion0.6.3• NDNAndroidversion0.6.5-3• BasedonthelatestversionofNFD(0.6.5)• IncludingupdatedGUIbasedonworkatNDNhackathon
• ndnSIM2.7• BasedonthelatestreleasedversionsofNS-3(version3.29)andNFD(version0.6.5)https://ndnsim.net/2.7/RELEASE_NOTES.html
• Mini-NDN0.40https://github.com/named-data/mini-ndn/releases/tag/v0.4.0
• Named-dataLinkStateRoutingProtocol(NLSR)version0.5.0https://named-data.net/doc/NLSR/0.5.0/RELEASE-NOTES.html
Recent New Code Releases / In progress • pSync,asynchronizationprotocolforNDN
https://named-data.net/doc/PSync/0.1.0/RELEASE-NOTES.html
• NDNIoTPackage• Mini-NDN-WiFi
NDN IoT Package • AnNDN-basedIoTframeworkwithtwogoals• Localizedtrustandautomatedsecuritymanagement• Ease-of-useIoTsoftwaredevelopmentkit
• Features• LightweightNDNsoftwarestackandforwarder,specificallytunedforconstraineddevices• Seamlessintegrationofheterogenouslinklayerprotocols(BLE,WiFi,IEEE802.15.4,etc.)• Ease-of-usehigh-levelAPIsforbootstrapping,servicediscovery,accesscontrol,andschematizedtrustmanagement
• EasyadaptationtonewIoThardware/softwareplatforms
• Ongoingefforts• Furthermemory-savingNDNforwarderdesign• DemonstrativeapplicationtoillustratehowtobuildanIoTsysteminafundamentallydifferentwayfromtoday’sIP-basedsolutions
• Moredetailedtobereported@nextIETF
8th NDN Hackathon (March 8-10, 2019) http://8th-ndn-hackathon.named-data.net/hacks.html
• FirstPrize:NFD-AndroidEnhancements• AlexAfanasyev,JuPan,SanjeevKaushikRamani,DavidePesavento
• SecondPrize:SigcommTutorialApp(NDN-IoTdemo)• ZhiyiZhang,XinyuMa,EdwardLu,YuGuan,Erynn-MariePhan,LaqinFan
• ThirdPrize• Self-LearningforAdHocWirelessNetworks• MdAshiqurRahman,DavidePesavento
• SyncinMANETLibrary+Demo• Tianxiang,Zhaoning,Spyros
• AddressingndncatchunksPerformanceIssues• KlausSchneider,SaurabDulal
NDN Project at Google Summer of Code • https://summerofcode.withgoogle.com/organizations/6559809451589632/
Publications/Presentations/Tech Reports • “ANoteonNamingandForwardingScalabilityinNamedDataNetworking”YuZhangetal,ICC2019Workshop,May2019
• “TheRoleofDataRepositoriesinNamedDataNetworking”LixiaZhangetal,ICC2019Workshop,May2019
• “ProofofAuthenticationforPrivateDistributedLedger“ZhiyiZhangetal,NDSS2019workshop,Feb2019
• “OntheGranularityofTrie-basedDataStructuresforNameLookupsandUpdates“ChavooshGhasemietal,toappearinACM/IEEETransactionsonNetworking2019
• “PacketForwardinginNamedDataNetworkingRequirementsandSurveyofSolutions”ZhuoLietal,toappearinIEEECommunicationsSurveys&Tutorials2019
• “AnOverviewofSecuritySupportinNamedDataNetworking”ZhiyiZhangetal,IEEECommunicationsMagazine,Nov2018.
SincelastIETF
How to learn more
• CommonClientLibrary(CCL)• C++:https://github.com/named-data/ndn-cpp• Python:https://github.com/named-data/PyNDN2• JavaScript:https://github.com/named-data/ndn-js• Java:https://github.com/named-data/jndn• C#(.NETFramework):https://github.com/named-data/ndn-dot-net
• PSync:ScalableName-basedDataSynchronizationforNamedDataNetworking• https://named-data.net/publications/scalable_name-based_data_synchronization/
• CommonNameLibrary(CNL)• C++:https://github.com/named-data/cnl-cpp• Python:https://github.com/named-data/PyCNL
• NDN-RTC:https://github.com/remap/ndnrtc