rm world 2014: mining financial markets with rapidminer
DESCRIPTION
TRANSCRIPT
For Finance
RAPIDMiner
New Order
Around 70 per cent of the orders to buy or sell on Wall Street
are now placed by quant driven software.
Information flow
Information flow is increasing but not not necessarily
Intelligence.
Efficient Market theory
EMT has dominated market theory but increasingly
behavior analysis plays a major part.
RapidMiner
Innovative products like RapidMiner have a role in this
financial ecosystem. Cutting edge and flexibility is key
relative to the established players.
WHAT’s CHANGED ?
Old Order
E
w
o
5
$
What
Drives
Global
economy
Fund
MANAGERS
K
Hedge
Fundsx% of the Market
HSovereign
Wealth
FundsxReal
Economy
(
Quant
Finance
bcommodities
demand
A
a
Gold
S
Hit View -> Show Presenter Notes
to view important information!
MarketsMarket
ANALYSTS
K M&A
Asset
ManagerSChina
$
US
FED
Financial
Media
K
central
banks
High
Frequency
Trading
Financial
Data
6
5
4
1
3
2Easy ?
Much of the data is still
unstructured. New tools
help
Open ?
Hardly but some
encouraging signs
Free ?
Some free data but much more
paid aggregation
Transparent ?
Where did this data come
from
Market Observations
Usage Rights
Free or paid data may be licensed
only. No redistributions rights.
Data Source ?
Much of the data used in financial
analysis originates externally to the
company reviewing
Finance and economics ExtensionEasy: Data operators pull financial data directly
into Rapidminer.
Live : Direct from source so data is not stale
Free & Open : No cost for using data
Unrestricted : Free to redistribute*
Multi-instrument : Stock, bond, indexes,
currencies, etc.
Transformation: Allow one to transform the data
into more usable financial formats
Analytics: Backtester which is essential to test
models
Sentiment: Search unstructured data
K
K
K
K
K
K
K
K
Open Market Data Initiative
Bloomberg Data
Open API : Free to use license. No free
access to subscription content
Open Identifiers : Bloomberg ID
Massive Aggrataor : news, exchange data,
reference data, etc
Markets: Global and covers all type of markets,
currencies, bonds, stocks.
Live: detail data
Restrictions: Not licensed to redistribute. Data
must stay local
K
K
K
K
K
K
Federal Reserve Economic Data
FRED Data
236,000 US and international Time Series
Sources : Central banks,corporations, US Gov,
Economic Institutions
Categories : Academic, Financial, and Economic
data.
Live: updated and refreshed daily
FREE: No restrictions on redistribution but you do
need to register for an API key
K
K
K
K
K
World DataBank
World Bank Data
8,000 : Data series
API : Well documented
Categories : Academic, Financial, and Economic
data.
Live: API update daily
FREE: No restrictions on usage and no api keys
necessary
K
K
K
K
K
EuroStat Statistics Database
EUROSTAT Data
5,300 : Data series
No API : Difficult to pull systematically
Categories : Government, Institutional, Financial,
and Economic data.
Live: update daily
EU: Mostly EU data.
FREE: No restrictions on usage and no api keys
necessary
K
K
K
K
K
K
TransformationsDifferencing : Use this operator to calculate
period-over-period change in a series.
Indexing : Use this operator to index a series to
another basis (e.g., Inflation Adjustment).
Replace Missing : Replace missing values for a
set of attributes. Useful in a merged financial
timer series where one needs to replace a
missing current value in a row with the previous
value.
REBASE: UUse this operator to convert one or
more series to a common basis.
K
K
K
K
Lagged/forward returns : Use this operator to
calculate lagged/forward security return.
cumulative SUMS: Use this operator to calculate
the cumulative sums of one or more attributes.
winsorize : This operator calculates the
percentiles of a numeric attribute and replaces
all values above and below those percentiles
with the respective percentile values.
Fincalcs: Use this operator to calculate the
cumulative sums of one or more attributes.
K
K
K
K
YOUTHANK