Download - VUG5: Varnish at Opera Software
![Page 1: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/1.jpg)
Varnish @ Opera
Varnish Users Group MeetingParis, 22nd March 2012
Cosimo Streppone <[email protected]>
![Page 2: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/2.jpg)
• October 2009• 1 old recycled machine, 2 Gb of disk allocated• Started serving static pictures (1M+ req/day)• Then more...• Even more...• ...• ~15% of all My Opera requests were «varnished»• Around 8M req/day
1st Varnish deployment: My Opera
![Page 3: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/3.jpg)
• Still using Debian EtchFirst Varnish instance was running v1.x from Etch.several years old, not good
• Experienced VIPs– ”Very Interesting Problems”– User X getting User Y's session– Random users getting admin powers. Nightmare!
• Theory: Varnish was caching response bodies that containedSet-Cookie: opera_session=<session_id>
My Opera – The start
![Page 4: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/4.jpg)
if (req.url ~ "^/community/users/avatar\.pl/[0-9]+$"
|| req.url ~ "^/.+/avatar\.pl$"
|| req.url ~ "^/.+/picture\.pl\?xscale=100$"
|| req.url ~ "^/desktopteam/xml/atom/blog/?$"
|| req.url ~ "^/desktopteam/xml/rss/blog/?$"
|| req.url ~ "^/community/api/users/friends\.pl\?user=.+$"
|| req.url ~ "^/community/api/users/groups\.pl\?user=.+$"
) {
unset req.http.Cookie;
unset req.http.Authorization;
lookup;
}
My Opera – The start
![Page 5: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/5.jpg)
...
# Check for cookie only after always-cache URLs
if (req.http.Cookie ~ "(opera_session|opera_persistent_)") {
pass;
}
# DANGER, Will Robinson! Caching the front-page
# At this point, lots of Google Analytics cookies will go in.
# No problem. It's stuff used by Javascript
if (req.url ~ "^/community/$") {
lookup;
}
pass;
}
My Opera – Pass logged in users
![Page 6: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/6.jpg)
My Opera: testing Varnish setup
...ok 289 - Got response from backend for /community/ (from ...) ok 290 - Correct status line# Adding header [Cookie] => [language=it]# ----------# GET http://cache01.my.opera.com:6081/community/# Host: my.opera.com# ------------ok 291 - 2nd request: got response from backend for /community/ (from...)ok 292 - Correct status line# X-Varnish: 1211283813 1211283812# X-Varnish-Status: hit# X-Varnish-Cacheable: yes, language cookie# X-Varnish-URL: /community/ok 293 - URL '/community/' was handled correctly by varnish# cookie_header:ok 294 - URL '/community/' has correct cookies (or no cookies)1..294
All tests successful.
X-Varnish: 1211283813 1211283812X-Varnish-Status: hitX-Varnish-Cacheable: yes, language cookieX-Varnish-URL: /community/
![Page 7: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/7.jpg)
My Opera – Next steps
![Page 8: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/8.jpg)
My Opera – Next steps
● Front page caching● Static assets and UGC● On-the-fly thumbnails● “Shields-up” configuration
![Page 9: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/9.jpg)
Problem
• Very dynamic, i18n• Accept-Language
header variation• Vary: Accept-Language sub-optimal
Front page caching
Solution
• varnish-accept-language “extension”
![Page 10: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/10.jpg)
Client sends
Accept-Language: ru, uk;q=0.9
Accept-Language: es-ES, es;q=0.8
Accept-Language: fr, it;q=0.7
Accept-Language: fr
Front page caching - Accept-Language
Backend receives
Accept-Language: ru
Accept-Language: es
Accept-Language: it
Accept-Language: ben
SUPPORTED_LANGUAGES = “:de:es:it:ru:”DEFAULT_LANGUAGE = “en”
![Page 11: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/11.jpg)
Front page caching
![Page 12: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/12.jpg)
Problem
• One central location• SPOF• High latency US -> NO
Static assets and UGC servers
Solution
• Decentralized varnish servers in multiple DC
• Talking to 1 backend• Very long TTL• Health probes• Cache invalidation API• Built our GeoDNS
![Page 13: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/13.jpg)
Problem
• Change of Design™ made our millions of pre-generated thumbnails useless
Thumbnail generation and caching
Solution
• Switch to on-the-fly generation model
• Used mod_dims (AOL)• Varnish on :80• 2 backends
300k objects95% hit rate avg800 req/s/backend peak
![Page 14: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/14.jpg)
Thumbnail generation and caching
How it works
http://localhost/dims/ crop/472x360/ contrast/+1/ quality/90/ /actual/picture/url.jpg (remote too!)
Using rewrite rules
Http://localhost/tn/small/ /actual/picture/url.jpg
![Page 15: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/15.jpg)
Thumbnail generation and caching
● Recognize mobile/non-mobile
● Scale thumbnails on the fly
● Reduce JPEG quality Ex.: /thumb/small/quality/80/some/path/pic.jpg
![Page 16: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/16.jpg)
Problem
• Original setup too specific to My Opera
• Long tail of non-popular content “unprotected”
• Can we find some more generic setup?
Shields-up configuration
Solution
• DDoS• Varnish in front, rather
than after frontends• Cache most logged out
requests with lower TTL• Compromise solution,
but generic enough
![Page 17: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/17.jpg)
Sitecheck
![Page 18: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/18.jpg)
• Used by the Opera browser• Must work! Failure not an option• ~8k req/s/backend peak• 2 varnish boxes, 16k req/s, 20k peak• 85% hit rate• TTL 10'
Sitecheck – Malware, fraud protection
![Page 19: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/19.jpg)
Opera TV Store
![Page 20: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/20.jpg)
Country-level ban
• Contract mandates that TV Store shouldn't be available in specific countries
• Country check in the backend means no caching is possible
• Implemented with varnish-geoip
![Page 21: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/21.jpg)
![Page 22: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/22.jpg)
Country-level bansub country_ban_list_check {
# Allow testing of country ban if (req.http.Cookie ~ "x_geo_ip_forced\s*=\s*country:..") { set req.http.X-Geo-IP = regsuball( req.http.Cookie, "^.*x_geo_ip_forced\s*=\s*(country:..).*$", "\1" ); log "Forced X-Geo-IP to '" req.http.X-Geo-IP "'"; }
# Block access to tvstore in these countries if (req.http.X-Geo-IP && req.http.X-Geo-IP ~ "^country:(C1|C2|C3|...)$") { log "Country ban"; error 750 "tvstore is not available in your country"; }}
sub vcl_recv { C{ vcl_geoip_country_set_header_xff(sp); }C call country_ban_list_check;}
![Page 23: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/23.jpg)
Brand + device TV detection
• Analyze User-Agent header
• Regex the hell out of it
• Send X-Brand, X-Device header to backend
• Fallback Device detection in the backend(for development, test setups, ...)
![Page 24: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/24.jpg)
VCL library
![Page 25: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/25.jpg)
accept-encoding.vcl
# STD: Deal with different Accept-Encoding formatssub accept_encoding_normalize {
if (req.http.Accept-Encoding) {
if (req.http.Accept-Encoding ~ "gzip") { set req.http.Accept-Encoding = "gzip"; }
elsif (req.http.Accept-Encoding ~ "deflate") { set req.http.Accept-Encoding = "deflate"; }
else { unset req.http.Accept-Encoding; }
}}
![Page 26: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/26.jpg)
accept-language.vclC{
/* * Accept-language header normalization * * - Parses client Accept-Language HTTP header * - Tries to find the best match with the supported languages * - Writes the best match as req.http.X-Varnish-Accept-Language * * http://github.com/cosimo/varnish-accept-language */
#include <ctype.h> /* isupper */#include <stdio.h>#include <stdlib.h> /* qsort */#include <string.h>
#define DEFAULT_LANGUAGE "en"#define SUPPORTED_LANGUAGES ":de:en:es-la:fr:fy:hu:ja:no:pl:pt-br:ru:sk:sq:sr:tr:uk:vn:xx-lol:zh-tw:"
…
![Page 27: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/27.jpg)
maintenance.vcl + {up,down}.shinclude "/etc/varnish/accept-encoding.vcl";
backend oopsy { .host = "10.20.21.22”; .port = "80";}
sub vcl_recv {
set req.backend = oopsy;
# Serve page from within Varnish. See vcl_error() if (req.url == "/ping.html") { error 700; }
call accept_encoding_normalize;
# Collapse URLs, so that we have just one cached object set req.url = "/maintenance-down";
remove req.http.Cookie; remove req.http.Authorization; return (lookup);}
![Page 28: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/28.jpg)
purge.vclacl purge { … }
sub vcl_recv {
if (req.request == "PURGE") { If (! (client.ip ~ purge)) { error 405 "Not allowed."; } purge("req.url == " req.url); error 200 "Purged."; }
else if (req.request == "PURGE_SUFFIX") { set req.http.X-URL = regsuball(req.url, "\[|\]|[\\^.$|()*+?{}]", "\\\0") "$"; purge_url(req.http.X-URL); unset req.http.X-URL; error 200 "Purged suffix."; }
else if (req.request == "PURGE_PREFIX") { … }
}
Ugly!
![Page 29: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/29.jpg)
X-forwarded-for.vcl
# See http://www.varnish-cache.org/trac/ticket/540sub inject_forwarded_for {
# Rename the incoming XFF header to work around a Varnish bug if (req.http.X-Forwarded-For) { # Append the client IP set req.http.X-Real-Forwarded-For = req.http.X-Forwarded-For ", " regsub(client.ip, ":.*", ""); } else { # Simply use the client IP set req.http.X-Real-Forwarded-For = regsub(client.ip, ":.*", ""); }}
Wat!?
![Page 30: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/30.jpg)
Testing VCLs – http-cuke
![Page 31: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/31.jpg)
http-cuke – csrf.test
Feature: TVStore uses cookies to protect against CSRF attacks
In order to protect the users from CSRF attacks As a TV Store developer I want to verify that some pages send out a CSRF cookie token to the browser or device
Scenario: Accessing the Backgammon application URL
Given a "Opera/9.80 (Linux … Opera TV Store)" user agent When I go to "https://tvstore.server/store/app/backgammon" Then the final HTTP status code should be "200" Then the page should contain "A board game for one player"
Then the page should not be cached by varnish
Then the server should send a CSRF token
![Page 32: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/32.jpg)
http-cuke – prove-like output
$ http-cuke --test ./csrf.test
$ http-cuke --test-dir ./some-dir
![Page 33: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/33.jpg)
http-cuke – a sample test run
# ============================================================# FEATURE: TV Store uses cookies to protect against CSRF attacks# ============================================================# ------------------------------------------------------------# SCENARIO: Accessing the Backgammon application URL# ------------------------------------------------------------ok 1 - Given a "Opera/9.80 (Linux...)" user agentok 2 - When I go to "https://tvstore.server/app/backgammon"ok 3 - Status code is 200 (expected 200)ok 4 - Then the final HTTP status code should be "200"ok 5 - String 'A board game for one player' was found in the pageok 6 - Then the page should contain "A board game for one player"ok 7 - X-Varnish header contains only current XID (523289525)ok 8 - Age of cached resource is zerook 9 - Then the page should not be cached by varnishok 10 - CSRF token was found (49a0da1b2758bf62a028072e4f7f36dc)ok 11 - Then the server should send a CSRF token
![Page 34: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/34.jpg)
![Page 35: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/35.jpg)
Puppet module
![Page 36: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/36.jpg)
varnish/manifests/init.pp
class varnish {
package { "varnish": ensure => "installed" }
file { "/etc/init.d/varnish": … }
file { "/etc/sysctl.conf": … } exec { "update-sysctl": … }
file { "/usr/share/varnish/purge-cache": … }
service { "varnish": ensure => "running", … }
munin::plugin::custom { "varnish_": } munin::plugin { [ "varnish_backend_traffic", "varnish_expunge", … }}
![Page 37: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/37.jpg)
Custom init script
# Lower stack limit demand for every Varnish thread# http://projects.linpro.no/pipermail/varnish-misc/2009-August/002977.html# Still relevant for Varnish 3 ??ulimit -s 256
# Startup with custom cc_command fails# Filed Debian bug #659005if bash -c "start-stop-daemon \ --start --quiet --pidfile ${PIDFILE} \ --exec ${DAEMON} -- -P ${PIDFILE} \ ${DAEMON_OPTS} > ${output} 2>&1"; then log_end_msg 0else …
![Page 38: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/38.jpg)
Custom sysctl settings
# From http://varnish.projects.linpro.no/wiki/Performance# + our own tweaking and tuning
net.ipv4.ip_local_port_range = 1024 65536net.core.rmem_max = 16777216net.core.wmem_max = 16777216net.ipv4.tcp_rmem = 4096 87380 16777216net.ipv4.tcp_wmem = 4096 65536 16777216net.ipv4.tcp_fin_timeout = 30net.core.netdev_max_backlog = 30000net.ipv4.tcp_no_metrics_save = 1net.core.somaxconn = 262144net.ipv4.tcp_syncookies = 1net.ipv4.tcp_max_orphans = 262144net.ipv4.tcp_max_syn_backlog = 262144net.ipv4.tcp_synack_retries = 2net.ipv4.tcp_syn_retries = 2
![Page 39: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/39.jpg)
Purge cache script
Modeled after Debian vcl-reload script
$ purge-cache -a$ purge-cache -u http://some.url$ purge-cache -r '^/(home|user)/'
![Page 40: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/40.jpg)
varnish/manifests/init.pp – 2define varnish::config ( $vcl_conf="default.vcl", $listen_address="", $listen_port=6081, $thread_min=400, $thread_max=5000, $thread_timeout=30, $storage_type="malloc", $storage_size="12G", $ttl=60, $thread_pools=$processorcount, $sess_workspace=131072, $cc_command="", $sess_timeout=3 ) {
file { "/etc/default/varnish": ensure => "present", owner => "root", group => "root", mode => 644, content => template("varnish/debian-defaults.erb"), require => Package["varnish"], notify => Service["varnish"], }
}
![Page 41: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/41.jpg)
Example of varnish::config
varnish::config { "cache-varnish-config": vcl_conf => "cache.vcl", storage_type => "malloc", storage_size => "20G", listen_port => 80, sess_workspace => 131072, ttl => 86400, thread_pools => 8, thread_min => 800, thread_max => 10000, # Necessary for GeoIP cc_command => 'exec cc -fpic -shared -Wl,-x \ -L/usr/include/GeoIP.h -lGeoIP -o %o %s',}
![Page 42: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/42.jpg)
varnish/manifests/init.pp – 3
define varnish::vcl ($source) {
file { "/etc/varnish/${name}.vcl": ensure => "present", owner => "root", group => "root", mode => 644, source => $source, require => Package["varnish"], notify => Service["varnish"], }
}
![Page 43: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/43.jpg)
![Page 44: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/44.jpg)
Migration to Varnish 3
![Page 45: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/45.jpg)
Following Debian stableNot there yet. Still anchored to 2.1
Migration 2.0 2.1 was relatively painless→
![Page 46: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/46.jpg)
Embedded C code?Migrate accept-language and geoip
extensions to VMODs
![Page 47: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/47.jpg)
2.1 3.0 syntax changes?→Test our VCLs
![Page 48: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/48.jpg)
CustomizationsAre they still relevant for 3.0?
(ulimit -s 256, etc...)
![Page 49: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/49.jpg)
I'd like to see in varnish...
![Page 50: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/50.jpg)
Easier VMODsIdeally, as easy as embedded C!
![Page 51: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/51.jpg)
varnishtop -t10s
Collect traffic for 10s and then report
Bonus feature: tags, AKA group-by
varnishtop -i RxURL -g RxHeader.Referer -t 60s
![Page 52: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/52.jpg)
Use headers as vars less than ideal
Introduce variables or registers to avoid
set req.http.X-Var = regsuball( req.http.Some-Header, '…', '\1');
![Page 53: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/53.jpg)
Better Cookies inspectionAvoid regsuball() mess on req.http.Cookie
set req.http.Cookie.SomeName = “xxx”;set req.http.X-Var1 = req.http.Cookie.sessionid;
and...
set var.SessionID = req.http.Cookie.sessionid;
![Page 54: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/54.jpg)
file storage?malloc works fine for us, but we always had
problems with file storage
![Page 55: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/55.jpg)
Better SSL handlingNot really. nginx works fine.
![Page 56: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/56.jpg)
Questions!
![Page 57: VUG5: Varnish at Opera Software](https://reader036.vdocument.in/reader036/viewer/2022081400/554bd25cb4c905ac708b4bc3/html5/thumbnails/57.jpg)
opera.com/jobs