importing and synchronizing content using feeds

Importing and syncing content using Feeds

Peter Vanhee2nd October 2011 - DrupalCamp Spain

drupal7

Who am I?

‣ Consumer and contributor to Drupal for over 5 years

‣ Co-founder Youth Agora @youthagora and Nuvole @nuvoleweb

‣ Web consultant in Barcelona @pvhee

www.europeancampus.eu

Content is aggregated and synchronized using Feeds

Use Cases

‣ Build Activity streams using feeds from Twitter, Facebook, ...

‣ Batch import sets of users from file

‣ Import Commerce Products

‣ ...

What is Feeds? 1

Feeds is...

‣ Import / aggregate framework

‣ Create and update drupal entities out of the box: nodes, users, taxonomy

‣ Real-time feeds support

Related

‣ Migrate‣ Table Wizard‣ User import‣ Node import

‣ Aggregator

Feeds as the standard for piping content

Feeds usage

Statistics from drupal.org

drupal 7

drupal 6

statistics from drupal.org

Fetcher

Parser

Processor

Fetcher

‣ Http

‣ File upload

A complete list is at http://drupal.org/node/856644

‣ CSV, RSS1, RSS2, Atom

‣ OPML, Sitemap XML

‣ XPath for XML/HTML

‣ JSONPath for JSON

‣ YouTube, Flickr, Slideshare

‣ iCal

Parser

A complete list is at http://drupal.org/node/856644

‣ Node

‣ User

‣ Taxonomy

‣ Commerce Products

Processor

A complete list is at http://drupal.org/node/856644

Example #12011.drupalcamp.es my drupal site

Import and sync RSS Feeds

Feeds importers are configurations of

Fetcher Parser Processor

create nodes of type Feed item

fields in Feed itemfields in RSS

And now... importing!

/import

We can do this with Drupal Core Aggregator too!

Let’s try spawning

User 1

Feed A

Feed B

Feed C

User 2 Feed X

User

User 1

Feed A

Feed B

Feed C

FeedsAggregator (core)

Spawning

Add Feed = Add Node

Example #2Import images

A node of type Story will hold a Flickr image

the Flickr XML feed

http://www.flickr.com/services/feeds/geo?format=rss_200&tags=drupalcamp

{

Parse XML with XPath expressions

//item

description

media:thumbnail/@url

geo:latgeo:long

titlelink

Fetcher

Parser

Processor

http

XPath Parser

Map targets from XPathNode processor

select XPath XML parser

source is XPath Result fields in Story

Importing

!

We have problems

Tags are not imported correctly

Feeds Tamper

http://drupal.org/project/feeds_tamper

Enter Tampering

Input Field Output Field

drupal feeds sevilla drupalfeedssevilla

Me</a>posted a photo:</p>

<a href=”.”>Me</a>posted a photo

explode

html_encode

Configure Tampering

More tamper actions

Add your own tamper plugins

snippet from html_entity_decode.inc

After importing... tags are now correctly split

Importing Big Stuff

‣ Feeds has Batch support

‣ Feeds can run on cron jobs in the background

Field support‣ Text‣ File‣ Image‣ Link‣ Geofield‣ Taxonomy‣ Password‣ Date‣ Integer

‣ Email‣ Multimedia‣ Address

Tip: check Feeds issue queues for patches

Feeds as a Framework

‣ Supports import/sync of Commerce Products (via Commerce Feeds)

‣ Straightforward to add custom fetchers, patchers and processors using Feed hooks (check feeds.api.php)

Finally... when not to use Feeds

‣ Content does not need to be stored by Drupal

‣ Complex content migration tasks

Are we there yet?

Now... Real-Time?

Publisher Subscriber

Are we there yet?

Are we there yet?

Are we there yet?

Yes. Here you are.

Slide taken from @alexb presentation on Feeds

Enter PubSubHubBub

Publisher SubscriberHub

What does this solve?1. Notifications to Subscribers managed by Hub2. Send only what is changed and when it gets changed

Euhm.. so what does Feeds do exactly?

Feeds makes Drupal a PSHB Subscriber

Update feeds in real-time from PSHB Hubs e.g. Blogger, Wordpress, Superfeedr, ...

Thanks!

Peter Vanhee@pvhee

Questions?

Peter Vanhee@pvhee