tweaking open source
Post on 16-Apr-2017
1.333 Views
Preview:
TRANSCRIPT
Slide 1
Tweaking Open-Sourcea case-study
Nelson Gomes (nelson.gomes@telecom.pt)Team Leader11th of November 2011
Introduction
OpenX Components
Improvements Introduced
Overall Architecture
Report Server
Problems Found
Links
Q&A
Talk Index
Introduction Online Advertisement
Online advertisement coupes with delivering ads;
Placing ads in sites is a complex process:
Obtain all ads electable for a placeholder;
Exclude ads with business limitations like capping;
Assure that the ads are beying presented to the target audience;
Assure the advertiser goals are being met;
Account the ads delivered;
Deliver the right ad format;
Introduction Online Advertisement
Examples
Introduction Online Advertisement
Examples
Introduction Online Advertisement
Examples
Introduction Online Advertisement
Examples
Introduction OpenX
Open Source advertising server;
Licensed under GNU General Public License;
Project forked from phpAds developed by Tobias Ratschiller in 1998;
Was called phpAdsNew, OpenAds and finally OpenX;
Features:
Has a web based GUI;
Extendable plugins architecture;
Serves ads throught JS and Iframes calls mainly;
Introduction OpenX
Support technologies:
PHP;
MySQL;
Web Server (Apache, Nginx);
Optional memcached usage;
Filesystem to serve ad content;
Introduction OpenDisplay
Starting from OpenX 1.8.5 version, SAPO OpenDisplay project began;
A four-person team started in April 2010 to analyse and improve OpenX capabilities to ensure entire SAPO's ad serving network;
In August 2010, OpenDisplay started to serve a major website, while development was undergoing;
In February 2011, SAPO began migrating it's ad serving network in a process that took about 3 months to complete;
Today OpenDisplay serves the entire SAPO's ad network;
Introduction OpenDisplay
OpenDisplay serves ads for several media:
Internet;
Mobile Internet;
Mobile Applications;
TV set-top boxes;
Connected TV's;
In the near future we'll be serving bulk campaigns for other media;
I'll try to tell in this presentation this endeavour steps and quirks;
OpenDisplay Components FrontendThis component is responsible to serve all ad formats;
No data processing is done here due to performance besides adserving itself;
The adserving is done using munged PHP scripts for performance;
Plugins are included in a on demand basis;
Database queries are cached;
So it's all about ad serving decision making;
OpenDisplay Components BackendComprises data feature processing;
Web based GUI for campaign and ad management;
Ad serving statistics;
Reporting;
Batch processing of ad delivery data for use by the frontends;
OpenDisplay Components TasksMaintenance Priority Engine (MPE)
Determines witch campaigns to serve given their priorities;
Calculates ad serving probabilites given it's probabilities and corrects them when underperforming or overperforming;
Maintenance Statistics Engine (MSE)
Processes ad serving numbers;
Starts and stops campaigns;
Improvements Introduced - General
Added reusable segmentation rules;
This way a rule can be reused in several campaigns;
Added compound segmentation rules;
Segmentation rules engine was rewritten, cause the previous segmentation system was inadequate;
Added the concept of Orders;
Sometimes a customer has several goals to different sites;
The concept of order allows to place several campaigns with different goals in a single customer order;
Improvements Introduced - General
Added Zone Groups;
Instead of selecting placeholders one a at a time we can associate several at once;
Imagine that a Run of Network (RON) campaign for all MREC (300x250) placeholders would need to be associated to all placeholders one by one;
Added revenue-share acounting;
For ads served on pages with third-party content;
This way, revenue can be shared with third-party content providers;
Improvements Introduced - General
OpenDisplay went through a security audit by SAPO's security team and several issues were solved;
Backoffice:
UI session cookies are now only delivered over SSL;
Session id generation function wasn't good enought and could be easily guessed. This correction minimized session hijacking;
New user profiles were added, and entity access was reviewed;
Some user profiles were changed to read-only, like advertisers and sites;
Improvements Introduced - General
Ads uploaded into the ad server are stored in a folder and served upon;
At first look there is no problem with this, but over time in some systems this can cause inode exhaustion;
So to prevent this, and speed up file retrieval we improved upload component to distribute the files in a two-level folder hierarchy;
OpenX can use a content farm to deliver ads, so we use this feature from the start;
Improvements Introduced - General
Traffic forecast:
OpenX doesn't have a traffic forecast engine, instead it uses an average of ads served;
We developed two alternative forecast algorithms using Python;
This forecast is critical for a couple of reasons:Inventory selling;
Correct impression allocation for campaigns, specially due to targetting rules;
Improvements Introduced - General
Traffic forecast example:
Improvements Introduced - General
Added data logging and analysis:
We started to summarize delivery properties to allow us to calculate precise segmentation delivery probabilies;
Using these numbers in combination with traffic forecast we can estimate the inventory for each campaign and it's overall probability of delivery;
Also, this information is useful to commercial purposes:Knowing the market is a very valuable information;
We are currently migrating some of this data to Hbase that reduces data, making it usable;
Improvements Introduced - General
Restructured VAST 1.0 system and upgraded it to 2.0;
Video Ad Serving Template (VAST) standard from Interactive Advertising Bureau;
Delivers video ads (pre, mid and postrolls);
Delivers overlays;
We also added a new type of ad that allows us to serve SAPO text ads has images;
This virtual ad type works has a proxy to a different ad system, combining two different ad systems;
Probably the first time an ad system combined them;
Improvements Introduced - General
OpenDisplay (...) http://(...)/video.flv
Improvements Introduced - General
Flash ads are a major problem in some systems that don't support Flash;
iPhones and iPads for example;
To assure these ads are at all times visible we added automatic Flash ad image generation to ads upload via Backend;
This way, even if a Flash ad doesn't have a fallback image, we generate one automatically;
This was accomplished using GNU's gnash in combination with xvfb-run that provides a virtual X Window System for gnash to run;
Improvements Introduced - General
Future developments will include bulk campaigns;These campaigns differ from regular campaigns cause we know the characteristics of the audience in advance;
Splitting audiance in sets with the same features we can process an entire set within the LP solver at once minimizing the number of variables;
So we can optimize the revenue using linear programming solutions;We will use GLPK (GNU Linear Programming Kit) has a solver to obtain an optimal solution;
This way we can provide a solution that maximizes a campaign's revenue;
Improvements Introduced - General
GLPK sample problem:
# Giapetto's problem, maximizing Giapetto's profitvar x1 >=0; /* soldier worths 3 */var x2 >=0; /* train worths 2 */
/* Objective function */maximize z: 3*x1 + 2*x2; // maximize Giapetto's profit
/* Constraints */s.t. Finishing : 2*x1 + x2
top related