Performance TuningFileMaker 10

December 4th 2009

Tim Neudecker

• Professional FMP Developer since ’91

• In House Developer for 10 years

• Founding Partner with Kyo Logic LLC

• 6 Proficiency Exam, 7-10 Certified

• Founder of NY and CT Dev Groups

What makes a solution Slow?

• Finds

• Calculations

• Slow Network

• Screen Draws

Let’s Tune your FileMaker Installation

• Ram Cache

• Records, scripts, layouts, etc also cached in RAM

• Saves re-reading records/data from temporary file on disk

• RAM cache defaults to 8MB in Pro, but can be much larger

• Tip: Solutions that work with large data sets may benefit from increasing file cache RAM size

• Effect is limited because the OS also caches recently used files in RAM, so “disk read” may get data from RAM anyway

• See file cache settings in Pro preferences Memory panel

Cache Setting in Pro

Cache Setting in Server

Stored Calculations

• Stored calculation fields are fast to view and use

• Downloaded at the same time as rest of the record

• Calculated only when a dependent field modified

• Unstored calculation fields are slower

• Must be recalculated every time displayed or used

• Can be really slow if they summarize many records/values or if calculation formula is extremely complex or recursive

Faster Calculations

• Short Circuit

• Calc Engine stops evaluating when logic permits it

• Let()

Demo

• 9/11 Text mesg

• ShortCalc

• Gaynor-Minden

Wide vs Long Tables• Records downloaded to Pro clients on demand

• For viewing in a window, printing, exporting, etc.

• For processing in a script

• For sorting or summary calculation

• All non-container fields of a record are downloaded together

• Even if only one field is on current layout or any tab panel

• Even if only one field of child record is in a portal

Wide vs Long Tables

• Tip: Put fields with large amount of text data that are only used occasionally in a separate table

• 1-to-1 related record with large text field should only be used in a few limited layouts (in Form view, not Table or List view)

• Avoids downloading large amounts of data that are rarely used, speeding up client and network performance

Demo

• Articles

Container Fields

• Container field downloaded on demand when…

• That field is viewed in a window, printed, exported, etc.

• That field is referred to in a calculation or script

• Container field may contain multiple “streams”

• JPEG and GIF are FileMaker “native” formats

• Inserting any other graphic format will cause a JPEG to be created and stored as an additional stream in the field for cross-platform use

• File name, size, and data all stored as separate streams

Container Fields

• Tip: Pro 10 only downloads the streams it needs

• Older Pro versions downloaded all streams (slow!)

• Feature of client, works with any host version

• Just one or two streams downloaded to view field

• For image: size and either “native” format or JPEG

• For file enclosure: type list and file name (NOT the file data)

• All streams downloaded to export or modify field

Speaking of FMP 10

• Multiple records may be fetched at once by client, to reduce number of download requests to host

• Form View: 25 records (was 5 before Pro 9)

• List or Table View: count of visible records

• Portal: count of visible portal rows

• Larger pre-fetch for Sort / Export / Summary

• 5000 records (was 50 before Pro 9)

Speaking of FMP 10

• Uses multiple cores more efficiently

• Better Pre-Fetching of data

• Server Side Scripts

• Schedule consistency checks

Make sure client and server is up to date

Server Side Scripts with FMS 10

• No more robots!

Very Fast

• 22 minutes     FMP 9 client over network

• 12 minutes     FMP 9 client running on server

• 5  minutes     FMP 10 client on Server

• 3  minutes     FMS 10 server script

Index

• Value index on a field is required for

• Relational Join (finding records in related tables)

• Unique / Existing data validation

• Value list (non-custom and non-ESS)

• Auto-completion

• Insert from Index (but ‘individual words’ requires word index)

• Find may use word or value index

• Only text field can have word index

Index

Index

• Index on a field can be disabled, but I don’t recommend it

• If the field is in a place where a user can do a Find on it, someone will, so let FMP create the index automatically

• If index disabled or Find criteria prevents use of a field’s index, all records must be scanned

• Scan takes longer the more records there are

• Using index is almost instantaneous compared to scan

Index

• Indexes can also make things slower

• when ever a record is created all indexes in the table must be updated

• As record count increase so does time to insert new records

Index

• Resetting all indexes

• Developers use fields a user may never see

• Recover command

Index Resetting with Recover command

Word Index

• Find on text field normally uses word index only

• Find on text does NOT use index with some criteria, usually making the find a lot slower…

Value Index

• Find on non-text field uses value index only

• Find on non-text does NOT use index with some criteria, usually making the find a lot slower…

Find Optimization

• Within each Find request, criteria on same table are optimized so fastest operation done first

• Index exact: companyID ==123456

• Index range: ownerFirstName =Fred (same as Fred*)

• Record scan: company =*Ma*er

• Since criteria on same table are ANDed together, set of records is narrowed down at each step so expensive scan work is minimized if possible

Multi table finds

• Use the Constrain and Find together

• Looping Omits

• GTRR from Related table

Demo

• Affinion - MyTasks v2

RelationshipsIndex Size

• Field data type can affect Join speed

• Each digit in Number field -> ½ byte in index entry

• Each character in Text field -> 2 bytes in index entry

• Larger index entries -> Larger index to read from disk

• Field size can matter with large record count

• Larger field size -> Larger index entries -> Larger index…

• Measurable at 1 million records on fast hardware, with match fields of 14 character Text vs 7 digit number

Relationships

• Q: Do multiple predicates speed or slow Join?

• A: It depends on operators in predicates and your data, since some operators will cause fewer rows to join and thus be faster

• Equi-join (= or ≠ operator) is fastest

• Range (a < MyField AND MyField < b) next fastest

• Greater or lesser alone usually slower

References

• 2009 Under the Hood Presentation by Jon Thatcher

• Some FileMaker Knowledge Base articles:

• #2984: Tips for Designing Networked or Shared Databases

• #5268: Performance Optimization of FileMaker Databases