orms: strengths, weaknesses, and building a - postgresql wiki
TRANSCRIPT
![Page 1: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/1.jpg)
ORMs: Strengths, Weaknesses, and Building a PostgreSQL-specific ORM
Jonathan S. KatzNovember 7, 2009 - PGDay.eu
![Page 2: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/2.jpg)
Outline
• Overview & Dirty Work
• The “ORM Patterns”
• Seriously, Why?
• Dissecting & Discussing ORM Features
• Peeking at some code
• Scalability & the PgSQL ORM
![Page 3: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/3.jpg)
Nota Bene
• I like working with SQL
• ORMs are another tool for solving problems, not necessarily the tool
![Page 4: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/4.jpg)
Overview: We’re Jumping In
• Let’s build an address book!
![Page 5: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/5.jpg)
Requirements
• Multi-user
• Store first name, last name, location
• Searchable / Sortable by location
• (Pretend this is a web app)
![Page 6: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/6.jpg)
Example 1: Naïvely We Commence
![Page 7: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/7.jpg)
Notes:
• Had to handle “SQL semantics” every-time
• Reusable? (or too contrived)
• Ruby “Hash” objects: no errors if no value for a key
• Also, isn’t Ruby object-oriented?
![Page 8: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/8.jpg)
Try #2: Make it “Developer Friendly”
• Make it geared towards my development language (in this case Ruby)
• Use Ruby to encapsulate SQL
![Page 9: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/9.jpg)
We Try Again
![Page 10: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/10.jpg)
Notes
• More code to setup, but less to accomplish our tasks
• Reading / Writing data much more familiar to developer
![Page 11: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/11.jpg)
Eureka!
• We made a simple object-relational mapper!
• Made our DB interactions more programmatic
• We could continue using this, but imagine if our domain were something like...
![Page 12: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/12.jpg)
![Page 13: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/13.jpg)
Formality Sake
• Object-relational mapping (ORM) - a programming technique for converting data between incompatible type systems in relational databases and object-oriented programming languages.(Source: http://en.wikipedia.org/wiki/Object-relational_mapping)
![Page 14: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/14.jpg)
Examples:• Ruby:
• ActiveRecord (http://www.rubyonrails.org)
• DataMapper (http://datamapper.org/)
• Sequel (http://sequel.rubyforge.org/)
• Python
• SQLAlchemy (http://www.sqlalchemy.org)
• PHP
• Propel (http://propel.phpdb.org)
• Doctrine (http://www.doctrine-project.org/)
![Page 15: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/15.jpg)
Two Notable Patterns
• Courtesy of and many thanks to Martin Fowler for the contents on the next two slides
• Source: “Patterns of Enterprise Application Architecture” (http://martinfowler.com/books.html#eaa)
![Page 16: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/16.jpg)
Active Record
• “An object that wraps a row in a database table or view, encapsulates the database access, and adds domain logic on that data.”
Source: http://www.martinfowler.com/eaaCatalog/activeRecord.html
![Page 17: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/17.jpg)
Data Mapper• “A layer of mappers that moves data between objects and a
database while keeping them independent of each other and the mapper itself.”
Source: http://martinfowler.com/eaaCatalog/dataMapper.html
![Page 18: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/18.jpg)
A Quick Interjection
• Why not combine parts of both?
• We will look at:
• ActiveRecord (eponymous)
• SQLAlchemy (data mapper)
![Page 19: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/19.jpg)
So, Why?
• Abstraction: Take SQL out of daily development (gasp!)
• Portability
• Development Speed
• Relationship management, or managing relations
• Expressibility via API
• Features!
SQL: like ASM for a developer talking to databases - nice to have it abstracted out
portability: can use with postgresql, mysql, mssql, oracle -- but still keep the same application code
development speed: abstracted a lot, less code to write; familiarity with an OSS ORM allows for quick turnaround on new projects
![Page 20: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/20.jpg)
Immediate Drawbacks
• Yet another layer of code
• Surrender some control
• Can be learning another language
![Page 21: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/21.jpg)
Design, Design, Design
• ORMs are no panacea: still need good software design
• API level
• Developer level
![Page 22: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/22.jpg)
ActiveRecord
• Core part of “Ruby on Rails”
• (Being overhauled for Rails 3)
• MVC = Model-View-Controller
• Let’s look at some code, then talk detailsExample 3!
![Page 23: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/23.jpg)
Interface Notes
• create, save vs. create!, save!
• true/false vs. Ruby exception -- consistency?
![Page 24: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/24.jpg)
Validations
• can test constraints before committing record to database
• can add custom validations if ActiveRecord does not provide one that fits
• returns a special “Error” object that can be parsed if there are failures (fun)
![Page 25: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/25.jpg)
Callbacks
• Trigger methods to run before, after initialize / create / save / update
• Useful for forcing data mutation or running special command
![Page 26: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/26.jpg)
Named Scopes
• Way of writing “programmatic SQL”
• In ActiveRecord, does not load data immediately
• Contact.begins_with(‘s’).visible.scoped(:conditions => { :created_at => Date.today })
• Note: if I used “ILIKE” suddenly code is not portable(!) - not “developer-proof”
![Page 27: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/27.jpg)
Which Reminds Me...
• Notice how ActiveRecord was smart enough to format the data types correctly?
• ActiveRecord also takes care of quoting and avoiding SQL injections *if used properly*
• User.all(:conditions => “name = #{name}”)
• User.all(:conditions => { :name => name})
![Page 28: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/28.jpg)
Associations
• has_many :blahs
• has_one :blah
• belongs_to :blah
• has_and_belongs_to_many :blahs
• (and many options for configuring these, e.g. :dependent => :destroy)
![Page 29: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/29.jpg)
Single-Table Inheritance
• class Friend < Contact
• belongs_to :user
• end
• ActiveRecord smart enough to reify appropriate class when User#contacts is called
• Note: Not using PgSQL’s built-in inheritance mechanism!
![Page 30: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/30.jpg)
Serialization
• Take a Ruby data type, store it, load it when record is reloaded
• serialize :data, MyOwnClass
![Page 31: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/31.jpg)
Transactions / Locking
• Supports PostgreSQL transactions (fakes it for some other databases)
• Can use locking for business logic purposes
• Occurs at application level
• Optimistic vs. Pessimistic
example #5
![Page 32: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/32.jpg)
Managing the Schema: Migrations
• Not part of the “ORM” per se, but worth a mention
• Helps keep track of “state” of the database and reproduce / tear down as needed
example 7
![Page 33: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/33.jpg)
This sound too good...
• You’re right!
• Well, you’re also wrong. But you’re partially right, there are some issues.
![Page 34: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/34.jpg)
How is the SQL generated?
• In ActiveRecord: all over the place
• Since can use multiple database adapters, does include database-specific SQL
• Examples for PostgreSQL
![Page 35: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/35.jpg)
More on ActiveRecord SQL
• Sometimes, ORM cannot do it all, e.g. a complex sort
• Model.find_by_sql
• Lose out on “Scope” features when using this method, e.g. fast pagination example 8
![Page 36: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/36.jpg)
Other “Missing Features”
• Cursors: ActiveRecord either loads everything, nothing, or in X batches
• Datatypes and Modules: e.g. XML, hstore
• Uniqueness / Constraint Errors: Treated as “SQL Error
• Functional indexes
• Inheritance / Partitions
• Functions (a lot of them)
• Prepare / Execute ?
![Page 37: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/37.jpg)
Shifting Gears: SQLAlchemy
• uses “data mapper” pattern
• separates SQL generation from actual objects
• I will give more of an overview due to less-familiarity with Python / SQLAlchemy
• Source: http://www.sqlalchemy.org/docs/05/ormtutorial.html
![Page 38: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/38.jpg)
Noteworthy Points
• Separates SQL generation from mapper itself (sounds familiar?)
• SQL generation: “not necessarily clean, but programmatic”
![Page 39: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/39.jpg)
Introducing “Postgresina”
• So pre-alpha it’s Ω
• Main idea: build on ORM towards specific PostgreSQL features
![Page 40: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/40.jpg)
Ideas
• Prepare / Execute - implicitly (hard) and explicitly (easy[-ier])
• Inheritance - table-wise vs. string-column / index and compare performance differences
• Easier to access different data-types / methods
• Loading into memory: when it’s time.
![Page 41: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/41.jpg)
Starting-Points
• SQL-generator:
• Must have solid API
• Must be accessible to developer Idea: want to be programmatic
![Page 42: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/42.jpg)
Scalability
• Want to remain programmatic and scale
• Be able to maintain roles for both developer + DBA
![Page 43: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/43.jpg)
Conclusions
• ORMs can help developers start-off quickly
• Issues with scalability + taking advantage of PostgreSQL features
• Should be possible to extend ORM functionality, but will not completely remove the need for SQL
![Page 44: ORMs: Strengths, Weaknesses, and Building a - PostgreSQL wiki](https://reader030.vdocument.in/reader030/viewer/2022021006/62038bcfda24ad121e4aabed/html5/thumbnails/44.jpg)
Questions?
• Now
• Twitter: jkatz05
• github.com/jkatz/postgresina