Hogyan k\u00f6zel\u00edts\u00fck meg azt a probl\u00e9m\u00e1t, hogy egy managed alkalmaz\u00e1sb\u00f3l naponta t\u00f6bb milli\u00f3szor el szeretn\u00e9nk \u00e9rni m\u00e1sik szerveren lev\u0151, viszonylag ritk\u00e1n, p\u00e1r percenk\u00e9nt friss\u00fcl\u0151 f\u00e1jlokat, a lehet\u0151 leghat\u00e9konyabb m\u00f3don?<\/p>\n
El\u0151sz\u00f6r is \u00e9rdemes tudni, hogy a workstation service, a windows f\u00e1jok t\u00e1voli el\u00e9r\u00e9s\u00e9nek kliense cache-el, az OS File Cache-\u00e9t haszn\u00e1lva, ami a helyi f\u00e1jlokat is cache-eli. Ha azonban m\u00f3dos\u00edtj\u00e1k a megoszt\u00e1son lev\u0151 f\u00e1jlt, kikapcsolja a cache-el\u00e9se erre a f\u00e1jlra, eg\u00e9szen addig, m\u00edg \u00fajra meg nem nyitjuk. B\u0151vebben majd egy teljes cikkben is megeml\u00e9kezek err\u0151l, amikor a Windows Cache r\u00e9szleteir\u0151l fogok \u00edrni (a technet site-on).<\/p>\n
Sz\u00f3val kapuk valamennyi teljes\u00edtm\u00e9nyn\u00f6veked\u00e9st az OS cache miatt, de mivel a f\u00e1jl olvas\u00e1sa sor\u00e1n a PInvoke-nak \u00e1t kell mozgatni az adatokat a nat\u00edv oldalr\u00f3l a managed oldalra, jelent\u0151s vesztes\u00e9geket \u00e9l\u00fcnk meg azzal szemben, ha inprocess, managed mem\u00f3ri\u00e1ban tudn\u00e1nk tartani a f\u00e1jlok tartalm\u00e1t, de ki\u00fctve \u0151ket onnan, ha megv\u00e1ltoznak.<\/p>\n
Gondolom a legt\u00f6bben m\u00e1r hallottak a System.Web.Caching.CacheDependency t\u00edpusr\u00f3l. Ezt alapvet\u0151en webalkalmaz\u00e1sok haszn\u00e1lj\u00e1k, de 2.0 \u00f3ta m\u00e1r egy\u00e9b appokban is haszn\u00e1lhat\u00f3. A seg\u00edts\u00e9g\u00e9vel b\u00e1mulatosan egyszer\u0171en lehet cache-elni a felolvasott f\u00e1jlok tartalm\u00e1t.<\/p>\n
object o = HttpRuntime.Cache[cacheKey];
\nif (o != null)
\n{
\n string s = (string)o;
\n}
\nelse
\n{
\n CacheDependency d = new CacheDependency(file);
\n string s = ProtectedRead(file);
\n HttpRuntime.Cache.Add(cacheKey, s, d,
\n Cache.NoAbsoluteExpiration,
\n Cache.NoSlidingExpiration, CacheItemPriority.Normal, null);
\n Console.WriteLine(“Content (re)cached”);
\n}<\/p>\n
Az al\u00e1bbi k\u00f3d egy teljes, futtathat\u00f3 teszt, ami a demonstr\u00e1lja a direkt f\u00e1jlolvas\u00e1s (implicit OS cache), \u00e9s a k\u00e9zi cache-el\u00e9s k\u00f6z\u00f6tti teljes\u00edtm\u00e9nyk\u00fcl\u00f6nbs\u00e9get:<\/p>\n
using System;
\nusing System.Diagnostics;
\nusing System.IO;
\nusing System.Runtime.InteropServices;
\nusing System.Threading;
\nusing System.Web;
\nusing System.Web.Caching;<\/p>\n
class Program
\n{
\n const string file = @”c:\\temp\\a.log”;
\n const string cacheKey = @”alog”;
\n const int loopLen = 50000;
\n private static HttpRuntime httpRuntime;
\n const int HRSharingViolation = -2147024864;<\/p>\n
static void Main(string[] args)
\n {
\n httpRuntime = new HttpRuntime();<\/p>\n
Thread t = new Thread(FileChanger);
\n t.IsBackground = true;
\n t.Start();<\/p>\n
int cc = GetSumGCCount(); L\u00e1ssuk a program kimenet\u00e9t, azaz az eredm\u00e9nyeket:<\/p>\n [source=’c’] L\u00e1that\u00f3, hogy a saj\u00e1t cache nagyon sokat gyors\u00edt, r\u00e1ad\u00e1sul a mem\u00f3ri\u00e1t is sokkal jobban k\u00edm\u00e9li. Val\u00f3j\u00e1ban annyira gyors a cache-el\u0151s eset, hogy az \u00edr\u00f3 sz\u00e1lnak nincs is el\u00e9g ideje m\u00f3dos\u00edtani a f\u00e1jt, ami amiatt fontos, hogy l\u00e1ssuk, m\u0171k\u00f6dik-e a cache invalid\u00e1l\u00e1s, azaz \u00e9rz\u00e9keli-e a cache, hogy m\u00f3dosult a f\u00e1jl. Ha az iter\u00e1ci\u00f3k sz\u00e1m\u00e1t felvessz\u00fck a p\u00e9ld\u00e1ban 5000-re, akkor m\u00e1r lesznek olyan esetek, amikor a CacheDependency \u00e9szreveszi a f\u00e1jv\u00e1ltoz\u00e1st:<\/p>\n Rem\u00e9lem tanuls\u00e1gos volt, de ha ben\u00e9ztem valamit, \u00e9s nem \u00e9rv\u00e9nyes a teszt, sz\u00f3ljatok, jav\u00edtom.<\/p>\n","protected":false},"excerpt":{"rendered":" Hogyan k\u00f6zel\u00edts\u00fck meg azt a probl\u00e9m\u00e1t, hogy egy managed alkalmaz\u00e1sb\u00f3l naponta t\u00f6bb milli\u00f3szor el szeretn\u00e9nk \u00e9rni m\u00e1sik szerveren lev\u0151, viszonylag ritk\u00e1n, p\u00e1r…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8,70,4],"tags":[],"class_list":["post-803","post","type-post","status-publish","format-standard","hentry","category-net","category-optimalizalas","category-szakmai-elet"],"_links":{"self":[{"href":"https:\/\/soci.hu\/blog\/index.php\/wp-json\/wp\/v2\/posts\/803","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/soci.hu\/blog\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/soci.hu\/blog\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/soci.hu\/blog\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/soci.hu\/blog\/index.php\/wp-json\/wp\/v2\/comments?post=803"}],"version-history":[{"count":2,"href":"https:\/\/soci.hu\/blog\/index.php\/wp-json\/wp\/v2\/posts\/803\/revisions"}],"predecessor-version":[{"id":811,"href":"https:\/\/soci.hu\/blog\/index.php\/wp-json\/wp\/v2\/posts\/803\/revisions\/811"}],"wp:attachment":[{"href":"https:\/\/soci.hu\/blog\/index.php\/wp-json\/wp\/v2\/media?parent=803"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/soci.hu\/blog\/index.php\/wp-json\/wp\/v2\/categories?post=803"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/soci.hu\/blog\/index.php\/wp-json\/wp\/v2\/tags?post=803"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\n Stopwatch w = Stopwatch.StartNew();
\n for (int i = 0; i < loopLen; i++)\n {\n TestWithoutCaching();\n ShowProgress(i);\n }\n w.Stop();\n Console.WriteLine(\"Elapsed time: {0}, collections during cached test: {1}\",\n w.Elapsed, GetSumGCCount() - cc);\n\n cc = GetSumGCCount();\n w = Stopwatch.StartNew();\n for (int i = 0; i < loopLen; i++)\n {\n TestWithCaching();\n ShowProgress(i);\n }\n w.Stop();\n Console.WriteLine(\"Elapsed time: {0}, collections during noncached test: {1}\",\n w.Elapsed, GetSumGCCount() - cc);\n }\n\n private static void ShowProgress(int i)\n {\n \/\/Console.WriteLine(i);\n \/\/if (i % (loopLen \/ 100) == 0) Console.WriteLine(\"{0}%\", i * 100 \/ loopLen);\n }\n\n static void FileChanger(object state)\n {\n while (true)\n {\n string s = ProtectedRead(file);\n ProtectedWrite(file, s);\n\n Console.WriteLine(\"File has been modified sucessfully.\");\n Thread.Sleep(100);\n }\n }\n\n private static int GetSumGCCount()\n {\n GC.Collect();\n GC.WaitForPendingFinalizers();\n\n int cc = 0;\n for (int gen = 0; gen < GC.MaxGeneration; ++gen)\n {\n cc += GC.CollectionCount(gen);\n }\n return cc;\n }\n\n private static void TestWithoutCaching()\n {\n string s = ProtectedRead(file);\n for (int i = 0; i < s.Length; i++)\n {\n char c = s[i];\n }\n }\n\n private static void TestWithCaching()\n {\n object o = HttpRuntime.Cache[cacheKey];\n if (o != null)\n {\n string s = (string)o;\n for (int i = 0; i < s.Length; i++)\n {\n char c = s[i];\n }\n }\n else\n {\n CacheDependency d = new CacheDependency(file);\n string s = ProtectedRead(file);\n HttpRuntime.Cache.Add(cacheKey, s, d,\n Cache.NoAbsoluteExpiration,\n Cache.NoSlidingExpiration, CacheItemPriority.Normal, null);\n Console.WriteLine(\"Content (re)cached\");\n }\n }\n\n static string ProtectedRead(string file)\n {\n string s = null;\n bool retry = false;\n do\n {\n try\n {\n s = File.ReadAllText(file);\n break;\n }\n catch (IOException ex)\n {\n int hr = Marshal.GetHRForException(ex);\n if (hr == HRSharingViolation)\n {\n retry = true;\n Console.WriteLine(\"File is locked while reading, retrying...\");\n RandomSleep();\n }\n else\n {\n throw;\n }\n }\n\n } while (retry);\n return s;\n }\n\n static void ProtectedWrite(string file, string content)\n {\n bool retry = false;\n do\n {\n try\n {\n File.WriteAllText(file, content);\n break;\n }\n catch (IOException ex)\n {\n int hr = Marshal.GetHRForException(ex);\n if (hr == HRSharingViolation)\n {\n retry = true;\n Console.WriteLine(\"File is locked while writing, retrying...\");\n RandomSleep();\n }\n else\n {\n throw;\n }\n }\n\n } while (retry);\n }\n\n private static void RandomSleep()\n {\n int r = new Random().Next(30, 200);\n Thread.Sleep(r);\n }\n}\n[\/source]\n\nHogy realisztikus legyen a teszt egy k\u00fcl\u00f6n sz\u00e1l \u00edrogatja is a k\u00e9rd\u00e9ses f\u00e1jt, ami az egyszer\u0171s\u00e9g kedv\u00e9\u00e9rt most egy lok\u00e1lis, 256ks \u00e1llom\u00e1ny.\nAz \u00edr\u00f3-olvas\u00f3 folyamatok persze id\u0151nk\u00e9nt \u00f6sszeakadnak, ez\u00e9rt a hibakezel\u0151 \u00e9s ism\u00e9tl\u0151 logika a k\u00f3dban. B\u00e1r el\u00e9g sok mesters\u00e9ges k\u00e9sleltet\u00e9s van a p\u00e9ld\u00e1ban, \u00edgy is el\u00e9gg\u00e9 marakodik a k\u00e9t oldal. Nem csak az id\u0151ket m\u00e9rem a k\u00f3dban, hanem a szem\u00e9tgy\u0171jt\u00e9sek sz\u00e1m\u00e1t is a k\u00e9t esetben. A direkt f\u00e1jlolvas\u00e1s eset\u00e9n minden egyes olvas\u00e1skor \u00fajabb \u00e9s \u00fajabb buffert allok\u00e1lunk, ami persze jelent\u0151s mem\u00f3riak\u00e9nyszert okoz. Hab\u00e1r lehetne mondjuk valami thread-local buffert el\u0151re allok\u00e1lni \u00e9s \u00fajrahasznos\u00edtani, a legt\u00f6bben \u00fagyis ezt a kev\u00e9sb\u00e9 hat\u00e9kony k\u00f3dot haszn\u00e1lj\u00e1k, \u00edgy realisztikus a teszt.\nA cache-el\u0151s p\u00e9lda eset\u00e9n egyszer\u0171en csak kapunk egy referenci\u00e1t a mem\u00f3ri\u00e1ban m\u00e1r benn lev\u0151 f\u00e1jltartalomra, \u00edgy nem t\u00f6rt\u00e9nik nagym\u00e9ret\u0171 mem\u00f3riafoglal\u00e1s sem. \u00c9rdemes tudni, hogy a nagyobb mem\u00f3riatartalmakat a CLR a Large Object Heap<\/a>-en t\u00e1rolja, amely kev\u00e9sb\u00e9 hat\u00e9kony mint a sima heap, sz\u00f3val \u00e9rdemes \u00e9sszel \u00e9lni vele.<\/p>\n
\nFile is locked while reading, retrying…
\nFile has been modified sucessfully.
\nFile has been modified sucessfully.
\nFile is locked while writing, retrying…
\nFile is locked while writing, retrying…
\nFile is locked while writing, retrying…
\nFile is locked while writing, retrying…
\nFile is locked while reading, retrying…
\nFile has been modified sucessfully.
\nFile is locked while reading, retrying…
\nFile has been modified sucessfully.
\nFile is locked while writing, retrying…
\nFile is locked while writing, retrying…
\nFile is locked while writing, retrying…
\nFile is locked while writing, retrying…
\nFile is locked while writing, retrying…
\nElapsed time: 00:00:01.2796481, collections during noncached test: 252
\nContent (re)cached
\nElapsed time: 00:00:00.0826961, collections during cached test: 2<\/p>\n\r\nFile has been modified sucessfully.\r\nContent (re)cached\r\nFile has been modified sucessfully.\r\nContent (re)cached\r\nFile has been modified sucessfully.\r\nContent (re)cached\r\nFile has been modified sucessfully.\r\nFile is locked while reading, retrying...\r\nFile has been modified sucessfully.\r\nContent (re)cached\r\nContent (re)cached\r\nFile has been modified sucessfully.\r\nFile is locked while reading, retrying...\r\nFile has been modified sucessfully.\r\nContent (re)cached\r\nContent (re)cached\r\nElapsed time: 00:00:00.7075406, collections during cached test: 5\r\n<\/pre>\n