From b37a1d43acaa752578809a816fc2bdba59413143 Mon Sep 17 00:00:00 2001 From: Chris Abraham Date: Thu, 5 Dec 2024 16:36:32 -0800 Subject: [PATCH 1/4] Add blog post "vLLM Joins PyTorch Ecosystem: Easy, Fast, and Cheap LLM Serving for Everyone" Signed-off-by: Chris Abraham --- _posts/2024-12-05-vllm-joins-pytorch.md | 83 ++++++++++++++++++++++++ assets/images/vllm.png | Bin 0 -> 32776 bytes 2 files changed, 83 insertions(+) create mode 100644 _posts/2024-12-05-vllm-joins-pytorch.md create mode 100644 assets/images/vllm.png diff --git a/_posts/2024-12-05-vllm-joins-pytorch.md b/_posts/2024-12-05-vllm-joins-pytorch.md new file mode 100644 index 000000000000..8a8f34a4ced0 --- /dev/null +++ b/_posts/2024-12-05-vllm-joins-pytorch.md @@ -0,0 +1,83 @@ +--- +layout: blog_detail +title: "vLLM Joins PyTorch Ecosystem: Easy, Fast, and Cheap LLM Serving for Everyone" +--- + +![vllm logo](/assets/images/vllm.png){:style="width:100%;display: block;max-width:400px; margin-left:auto; margin-right:auto;"} + +We’re thrilled to announce that the [vLLM project](https://github.com/vllm-project/vllm) has become a PyTorch ecosystem project, and joined the PyTorch ecosystem family! + +Running large language models (LLMs) is both resource-intensive and complex, especially as these models scale to hundreds of billions of parameters. That’s where vLLM comes in — a high-throughput, memory-efficient inference and serving engine designed for LLMs. + +Originally built around the innovative [PagedAttention algorithm](https://arxiv.org/abs/2309.06180), vLLM has grown into a comprehensive, state-of-the-art inference engine. A thriving community is also continuously adding new features and optimizations to vLLM, including the following: + + + +* [Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve](https://www.usenix.org/conference/osdi24/presentation/agrawal) +* [Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving](https://arxiv.org/abs/2407.00079) +* [Llumnix: Dynamic Scheduling for Large Language Model Serving](https://arxiv.org/abs/2406.03243) +* [CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving](https://blog.vllm.ai/2024/07/25/lfai-perf.html#:~:text=CacheGen%3A%20KV%20Cache%20Compression%20and%20Streaming%20for%20Fast%20Large%20Language%20Model%20Serving) +* [vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention](https://blog.vllm.ai/2024/07/25/lfai-perf.html#:~:text=vAttention%3A%20Dynamic%20Memory%20Management%20for%20Serving%20LLMs%20without%20PagedAttention) +* [Andes: Defining and Enhancing Quality-of-Experience in LLM-Based Text Streaming Services](https://blog.vllm.ai/2024/07/25/lfai-perf.html#:~:text=Andes%3A%20Defining%20and%20Enhancing%20Quality%2Dof%2DExperience%20in%20LLM%2DBased%20Text%20Streaming%20Services) +* [SGLang: Efficient Execution of Structured Language Model Programs](https://blog.vllm.ai/2024/07/25/lfai-perf.html#:~:text=SGLang%3A%20Efficient%20Execution%20of%20Structured%20Language%20Model%20Programs) + +Since its release, vLLM has garnered significant attention, achieving over 31,000 GitHub stars—a testament to its popularity and thriving community. This milestone marks an exciting chapter for vLLM as we continue to empower developers and researchers with cutting-edge tools for efficient and scalable AI deployment. Welcome to the next era of LLM inference! + +vLLM has always had a strong connection with the PyTorch project. It is deeply integrated into PyTorch, leveraging it as a unified interface to support a wide array of hardware backends. These include NVIDIA GPUs, AMD GPUs, Google Cloud TPUs, Intel GPUs, Intel CPUs, Intel Gaudi HPUs, and AWS Neuron, among others. This tight coupling with PyTorch ensures seamless compatibility and performance optimization across diverse hardware platforms. + +Do you know you can experience the power of vLLM right from your phone? During this year’s Amazon Prime Day, vLLM played a crucial role in [delivering lightning-fast responses to millions of users](https://aws.amazon.com/cn/blogs/machine-learning/scaling-rufus-the-amazon-generative-ai-powered-conversational-shopping-assistant-with-over-80000-aws-inferentia-and-aws-trainium-chips-for-prime-day/). Across three regions, over 80,000 Trainium and Inferentia chips powered an average of 3 million tokens per minute, all while maintaining a P99 latency of less than 1 second for the first response. That means when customers opened the Amazon app and chatted with Rufus, they were seamlessly interacting with vLLM in action! + +vLLM also collaborates tightly with leading model vendors to ensure support for popular models. This includes tight integration with Meta LLAMA, Mistral, QWen, and DeepSeek models, plus many others. One particularly memorable milestone was the [release of LLAMA 3.1 (405B)](https://ai.meta.com/blog/meta-llama-3-1/). As the launching partner, vLLM was the first to enable running this very large model, showcasing vLLM’s capability to handle the most complex and resource-intensive language models. + +To install vLLM, simply run: + + +``` +pip install vllm +``` + + +vLLM is designed for both researchers and production-grade serving. + +To run vLLM as an OpenAI API compatible server, just use the Huggingface model ID: + + +``` +vllm serve meta-llama/Llama-3.1-8B +``` + + +To run vLLM as a simple function: + + +``` +from vllm import LLM, SamplingParams + +# Sample prompts. +prompts = [ + "Hello, my name is", + "The president of the United States is", + "The capital of France is", + "The future of AI is", +] +# Create a sampling params object. +sampling_params = SamplingParams(temperature=0.8, top_p=0.95) + +# Create an LLM. +llm = LLM(model="meta-llama/Llama-3.1-8B") +# Generate texts from the prompts. The output is a list of RequestOutput objects +# that contain the prompt, generated text, and other information. +outputs = llm.generate(prompts, sampling_params) +# Print the outputs. +for output in outputs: + prompt = output.prompt + generated_text = output.outputs[0].text + print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}") +``` + + +Open-source innovation is part of the vLLM’s DNA. Born out of a Berkeley academic project, it follows the legacy of other pioneering open-source initiatives such as BSD, which revolutionized operating systems in the 1980s. Other innovations from the same organization include [Apache Spark](https://github.com/apache/spark) and [Ray](https://github.com/ray-project/ray), now the standard for big data and AI systems. In the Gen AI era, vLLM serves as a platform dedicated to democratizing AI inference. + +The vLLM team remains steadfast in its mission to keep the project “of the community, by the community, and for the community.” Collaboration and inclusivity lie at the heart of everything we do. + +If you have collaboration requests or inquiries, feel free to reach out at [vllm-questions@lists.berkeley.edu](mailto:vllm-questions@lists.berkeley.edu). To join the active and growing vLLM community, explore our [GitHub repository](https://github.com/vllm-project/vllm) or connect with us on the [vLLM Slack](https://slack.vllm.ai). Together, we can push the boundaries of AI innovation and make it accessible to all. \ No newline at end of file diff --git a/assets/images/vllm.png b/assets/images/vllm.png new file mode 100644 index 0000000000000000000000000000000000000000..5b272828f49806f820b846a5c722b4764f12f5ef GIT binary patch literal 32776 zcmeFZ_dA^37d3t(M2HemB5H)_BuMm0{P5(tGBI=RbN1PLt+m${qWMILjN~Q>1VLmf$_j7@B9en3 zyiQ^w@Hdf@+Ct!uOU{o}w28sXhuAC_f>ktmoq(z~_-RZ*_9?pNrAdVx&SJ3$WMzK+ z7u!3&Xt^UbA>rk3+uITBut$+lf=_RPr!-51OxYESX|*Nqo>M>B8O2LrE&uQJ|90U2 ze>;G$kw6Ys9X)Y4Vj{a#)I`n@lawUQdi!>IU-_m_rgj{h=P7rrse!Jtf-Mrof$Y75 zP$+|U97<^@G+(=>$Gzy0yj+g)?%J1Xb)7mYlq|QCMA&H9Z2l}*4-Ou=Sf0SVle6;j2I349yHhd3>TK7!Oi*a&0 zO$PrnI6Q5X8M9usMYpMAgHbd*LI@JuOEFgvku#n&#&AD<6;`Vi5OKcjf3e&Ua-EZ$ z54^hC+E#2P+zRX_D!*%Tfk*xxLt=v*bg!KG5CexHwktPZ;v>e#i_8rWHk%>J-qmgs z7pvCD`k&i8nT#W@U_Ex|(kE#Dkqb-kkbIp^(!GOm=HkiA)|oLCyqPiCF?-`%{Z$xO z@b0q_noMmY3yY=tC(?(1;*9;B1C4wR`jt9g6zj`g3sGHPmKH#!xVM(%{~R8-t9Gcg zJHD@^FrH#GZ@2mDDXb;)@bu5;VRX{qu)t*bQQ5lh-C;Ke>dun6O*@Nt?_7Iy_HtE$ zr9oVFj1yDq?zmr^y+bHs{m5-=bAa4+;#lh1wayZ^WY=MXPW764D3a^eARYwC?F}H7 zzd3bqGsvFr$nQ#NNcCPv9@=BkJjj8v_%1IcA_0NJJ`oOamq09JojiaDP^$0AA7+W(#9}#ZiAE>{36g4&RY`JiA)vr2Rb|fifW@rR89G#LPICF7S(MuVncYKpk zIWj@m@?yd~-spiiZK$gU(PodYy~A-`t#tNQ@A1*`y12M_XW6h4Z|0HTcPJnr6~c!g zD(;LJr%m0g(hV&TR)1WG8Yn>cDR9$}zJIiuRq7M_T5RJ>4|MVlM5h8bN1S;`$~Fp} zS5WYzqqEat03m3bC{A&Z3tnC81=U-*larIuwAZ5phY_|D_Hn~Brck5^-@pBX&l?aB zQ+Af%_gfH9U0)%KgDqV%u(sRI(?x#u!D5b6Y$ZtHgIx*25wEPQ#QwODLk$<-m%-hm z3b7<(m9w2Fe{(l(t*QVkk6(Lxm6R369r8H63gp{}+4)hqdVmKhA444a2J|=f0f-A6&Zv?_&Do>w2J+mpS zJxUxOALkqNjFfobFzw@|qoc!8ZhR@=?;Ht898}yice3zRno9UM=CJq#imHD}xr^}? z6q;R)#!AdPN9^lPmmI$Iy!o^9o58_MR8-Vt>)4wO`kSW(ftU-Sj(M1nx^RgKmJ>s- zN*Jx!QD%@C(>%W$xVpM3`%!_-I%}Wx)OW9))7^HwjJs=%0wR7;Fb4{K6#l(LM!CH5 zs6T&hs1ruZ!%$=Te0RXw;=8`ywv$VkMQ3ceHp=I55Bt-gcWHy2w^8X zJ3Ag(maGjEMmeXne-|KxoSKrA@rdM-1`qiFPi974os56WM;gn7o;M4)$8KU^V9+qj zyalmS6s+L37Ve-;L|l~m4R6jfd6hW8gzH)qtF6(Gwk7a&!|e?2YPtPH*LZxD>GP z=--aQS&7U+%(h)&|7vdVPz?W4Lp;2^5v(vLN9AK^sG!d}vVuf?`QZ+~QPqLa9ej|Y zzW;Zn%`aO7u##xryUQgW_)c{(Ez{fpZfGdE)-5_CHRD2iX(EhXIJ_V~|J&D&%_~s! z{{sDhEf6Z^tDj&%=aJF-Hc9tD9fFS9J z|2@>TT`9rL%W%9)5g<~m2ea@*r#GQe$sd8P<`r7m6x#!`K9$GSdf5J-S zmBh!CjC1hgPXFx=4{>Qb_W)!<^6TRd)@OPF)0DyHakX;XxbT|o=;%o4{dzxChjpbq zLiXY$zNMwb*KXng6Co7IN-YP<0NcI01ea7<$N45=Ad@try-biU=SK=nuttO8lvinJ zXvXsN3hUlIxCFI-N+$Ss=GTh(La!{!1@t|y&xnauW@pRH%#2S2tMoYEZSBn|EM!p^ z#t&F{j5~F(H;|Ljh|47tEtra;V#}3YS?u|}tkQ-V-F*G1da$`B;^8kPNFdh6|GSYf z5*8PDOeb{@U)xjc5OUyImDVb9bA0;rX>TdfVLBUSQA6M@V%hV}*==i5oSS&rn7$zq zpEYL(JPpWCly8NoDwqu7IKb<=Lp|omA)yD~-|J*H$QgHW;Otaa2K^w` z(|?Hx>I;uzg#xo;sCxJJV5+Ku=BOK01EZrJ9#YKMsAbQr^b>NzTTpv2EC`b;N$K^TW%vH?I%~R&)=2+(vP_p(RezU)WT4p;AwSZ`7YX(a7FRsQIX~S`}ZyD4+d0kt1k!m=Oa*S%fXw1Be9wNqeqmrR)~g0e(}4g9 zD=s74NWklxA1x=hQZzhy1>-iTbXw#9g<)K5(BOI8FoKp(e`n(C>s8YNCeI7Dqm%#|94W9rWDs8xxaVy z$E3TvyVGGXm|tU@Y0fIBdh~sLeLIQ2e?K0-3T3eV_aylJoqrIQ**GJuK&8KwIwcf* zrL4rxP&gGm6aycaoaCFRaFCt<9>o0BN`k^`CWyt)r zzd~~NDjdI&Dkf0-V;m?%{6$4{Fz%e39D0*y&(5U>4f@a^tR8glOAX$GpbtAmAoi*5 zaJ|ie5S0b%=*M}exUw=4@ZKe5wCT#>;i0HCQdn*I;kM^aA#&I?Hm~eKIY~?p-|~`eEP2oKxNIZbY8a5)zeFv7kg62@)7N4O94gl{JX91 zcIAVvH`HBuRBcXqWk(c=GBYz%e*E}D;dmY#mez0eBQ5R4IC{HYw!wv&y=SRAG2&y( zi)Gw}|7eBekd`+3HCt3TF0c1y=PCrD|J4*_cK_o&$R#y~r1@{q@HqBX;Ph-~IY@_f zzUYpH_RrF?g(qKiO)nz9sBE7kkCKV7Nw-IePAkqkP5g8HQ% zY-B*j&;#Yw&PIBu`ei6NB>m8aKhC_pE$X)A`=(ihtdSlFhChGI;^X5rYUjfxOTfYO zqR}&0nX@N&NTqv)nJh&m-8RH(i;7-DE?%mr1slfq8kh97fwIN@=& znvE5r3K8>b`$HB;d^AlnvblfEb=xZtET7B@Q>;^f6{&mg-kR3a+{MVbA%a+h3inqdY0SD_N+m&5i zU3X-W_}8vE8tFZ-8+VBX4^Qtt?YX>zs{!7AV@_bt#1H!9w<}NPLi410Nt;g?KEHw3 z)8(E8fX@WQ>8_FQ;SiQl)PB;&?@MewFB0`UQoSD9^N&*nV$SA4$ z_3ST`8u0a=S^Lki(wKum4B!EUKaj{n#p1>iK1W8&o^8M5>Q8`Bgq9y!Uyxl2U{=5t zwhSp*vft1s4joDmD|mmAa9x`NPaL7PyaLEEIGmH;03iMb#T!#$KUK9I7Z>N(O_3K? zHXNOhQ0ct4(7qo7zZi`mh2URKB|%|Twg|itgSSIC&KL0Ri;y{NaJQ8OTgF24!#y)5 zI!V%3cOv2s^h)h_!Ep!+`7dk4WTSNbZU*B)sgl~fCN$TN ze(%?`_?{gYY*%!Ff~ca>p0HiB|HL08Ro@XK_bJiNj*dHL-%q}y;%aK7=s`3vWtBou z-M@2Sk3OB|3b5ao(l|Axw00Q9CnXM>%4MvW_Ocm!c$w+o$Bb?uS7ru)C|0w--n#>s zvel0;EY{x$LaejNPFC}EM&En+TS~}oyzE%b5)bkT`40p_EBBj%DEy04Q=h+QAws&- zM6{aE|Nb;nhrK|Hld?lw_1KG-uWW1tgCD17rI)2(v?c}y_FkEqejgkfR^@OwR$Vrq;0D4-)}`ZXdHJ-qF?dOwx7jkH6cY zbUh6kVvRkS7XVk`d8CO%4^$Nt$etRzUIp{1nOe~6_QK9m_@DT=xw$3HSo~H>*HY2Y zz%CAI$Vv~m&~A)%3f6x!F*3r)4~i=wWS1)|D|eqh5nr6md77=8*yGGPo*d%qt&SIf zdFa3bE^yeqSlzZ9cP2Y!5HnQEN=-ueB9Xka#Hv)TK|M*rbrV-T20>2Mk|?q>d|*3z z*8*e*9O{!ctajTR0xts09&tQ~ZL{$*YmLoSv;?uJ9;6J%RWt|+R(^fNbvcp|aOx^q zYHDhQbGJ<(It&*Wc^iP&C|I#Q2qKJLp|Nig4sZdQHZLah>dy~O95QkF^nXvq6~v~x zqN3vdxh_EWs%{&jQIY@*ZiAIfNgs{AT-MdqwKz2;ecI}1HjtsR_m<`SFCi}WjD-Cw za4QM${aeuphtLk-U4H={o{uNy5Bw#OC_cbC>OexJ14Kzs8hgI47^S?xIrny#?_cWc z%uxB9#q7Bd)t$8uK~)y#AO58%^dp0c)ixID1APms{7s`-a>d9iCA$0i(_oB%C z`P8!3%Q$1m0)u%Pj<6W^NM{5V){bS66R4@RhUS2md7c^XL5w za9?!FD`dAlnH4FayYK$N15dlsc6u#B{9pT05#omWcw50~(1!L;!3AcB9lSYhw5;c7s5 z%qll(s+kYLTKVqL_azWq(+uQ6zC(2r<&DumMPUG7G15{(BR3C;#Plk&M{flg#@bE5 z_4QA%Z)5aCUy81ykVJB5UmjqSUpvJ6Pl zD%*AFtt(lj86|WkKD)vEhUMQjg64v7B274>WAinC(IgH$eHt>>RZR#S6L&8~k~K7? zlt>so2#P+7s;a6m(@W9td1YT;%>69qj1mBGj0Z$7(~fsw|oRC3$-%2Pky_0`*FSt;5RYkG4K-4{iV+qy~)0Dt9&^t z6i?q4nTzZ|E}oNFX`+DS|4}69hC9$aM#xxX&bKx; zY5NahWwEEdwu7EsP=ckInbZT9mA)kO-i($$ z1UdhgTZhXz@M__#a_1sPE>45O_clU ztZL&Fqrm{cQVWJ9%)kkA@$+N+K-^Gv3d#D=ki5?*D%u&s;eVh;UHL6dD+j;lfBD~~ zfvlX-n-#Lp^!0xZ?VsOopLS&A=I5`Si!-zrZ*hs!-ih!(znz_vvx?v9sHSuc+&>Eo zU1q1YTkgD>U6=9yx3I{)vSGl{tR#p6>`VAU$8zML70hH|L2ByX{l>SKWjBHhu+=4e zsysYAX6U0$x-UJ^BoFK-J;shUCpb@LodKql@jc70Ip6D)eUL;56*+XxC<4$CpKLJD zH{$X^`+E$PtV;nNB$s#niP-G!w{PDp>XS}i0!%(rhiL_bl<`Bd9AbS3T;zhorZtMF zVapcW1wh7(Z=0)R_1<00l6!Y4!1tjBNU$uGXfnb1a=n*N1-vCLrbi!&w|?pQ%u_Z@ z7165R3RqVTqw?-tm~hVziG@h{h1txuTwx4-tp z^2ES%%gcvdu0m4_6?j@u$URV>W$9>WbWZL5j+WaQl-rKQD6#k+C=VJ$M-58|Ge?*# z1liOMBZP#6=0Me~R|hq6g9XQ~Zn$CBdvf+=qAhvW7FU1Mit}y-U+s!Lj)kjdm1dG; z#w;u@CWPaqqDRvRl<~5Y+z^g4b-q|ZY3Zqd*Ch?LRqb=Sx36ZY@>q1nXP?4-V|3R} zmRb&gcsv3kK1l!#^_bgwwP1wiqs;bImkdOe0I@p}hBO<$bFxTsUvHvmoBuaWm81RyRtiRSu)rfDuymG7G zoqs3Upa>Bss<7B)p>$SuygEU`+S4FqgA8sfkB&YE8k=w4&z~oE-WET7#78TPOJ)F> zJtjSQHo+4@z`TQ_7)xX~MXZWf8+UpDS%AZx`hbk-w^T8w774igCkK1`i*npGub&5) z@_ogW+k^=YoNUPAHvS-5JVoqtiqC}C&wSlpBa>Ck&#`M-x$`mth)}caoT@0oT&HzQ z2LGGZ_I@x0+R6gS9_=mdRFp0Vr;D2gctuKAAK>RK{zwn6y3Nbmxo-%11RCWIGk%Rg zM|+&m6V3Occ$8&*eSQC2a|b}hzm0F{xH92(abc&TERfefU6pEM9xgpgAy|EU;appz z3Fo9!Wa6fta_#-q4|}|fS|INn-IEhpl1W3!7%2?cPF&bHEI+v)XMSp@`kSI=2=YRUfJ0;7h2Xr>Ez13!TZiw#K;> z@_vi=|4Q4Hy^}eQqtDyxi7Z~7hR2@Bi-lEgO7|mN2HL#<8{ww ztF|m_qr_uRO|x!6(8~C6-El3NuhU=t#;(URT_mD~Rz>>nm}N?BeqG%(U9TC!=CnV; z&t(@Bez&hZ_>kD$3vVxQQU5L#3|z;eX^xi;bzaTSy*Pf{YLm2yswnuDtw_=oPHk4< zB6G_z!Q6l>M1>S;Y@%nCTNB?XYNCDeP0TR_hizP*8^mpl7B^L`W+jRhZbDtj(%vp2 zH>F1eTOvT@+XA$|#fZZ!m&0-OTY6zr-MAl9SgluI-{}4QCt^l_h>HN_;(y(UA9iNm zoayjb&5dc|DU60U6G26F6RZ)cJ3&RRA5f#Hrl5Jkg)j_HHu9dR{3Sm|HHB^**1H&$nyDgwsNNP;?WIHzcPrw!#5a1`SywDu zy0pvJ-tN6rT@HeU#Z@t%d8NRbe?U}PfH2|S7)rJfb-}I}KXc53L@BUc18S4Y^+O6) zSMJQ4p&vgcm+@iAESu*Ya|#OT{(#dM1A;*WP>9L*n_Q<{jc~-Ob;QIUqlCj`Ug6l} z8xXWZk3=0Fco&CYg?kOe{E;*l+MnQ|ycVUN_<#uoIN^1*W#W$%GADmYWWQkmmCMiP!=+%qS0cnvYAa6M zdrna{d^9uTo;7*)5#8LHIXtdW(Rut#gAeI_;UdR@_`N6^GE&S^4_4bPeGUvFP%WDl zAxWn8d_Rbwiho28$UdHc)iMtwEWh;3yI0Kkp>qI^U-vte4!?PSZ5x(zb5I;ysZJJm zdI<7a+V!srn!Y!Kp3#6UX7b;6(o$Dwe$r<&ILJmw@7S*9tf(5aFI%qmO{|#hUg5Y+ z*S2LJ8H{^gm;w({essk!?K>)2OH)&Gt>Rx~6%;T#0%Vi&LLxN98YvIdbG_Di$3ZOx z8-CENta6yC^Vj%mW3^d9v7WrEd6hK>hbW)ZA=dNh9VGovt@4cZ;Yfd-A@=t$1cjIe zwMGoJ`EQP?BZ;fmt&p-$Ci9NI%DEhr4d>?Mw7Xnwpa8u`@k>l6k66cT*}i!$B}|}? zw`E()?Ep*DgsJe+c8&wqkdRe+GUfn?h_EUQ_GCw1y}{Ke;jZbg4OPL~1E&A4@xiv& z*!?0 z#>Zp-4(XY6KS*kI0Ci60!Oc&}8=%?6l@=4N$#pQ@c$wuKfK*~omHy#A4 z)4bjBT10u81lZ*qYQUlpN)<|*yDv}c0u9s)ANXIFHz8tmD>qpk-fm-!(b4D z`f(iK+-FQm%-W$rgYzr^V?{yXeS2PXcAJ{Svz5`gixfvLR;o|g^x1LDsq?o~AP=d0Vff+9BsutMXE@AWOd#n@fw!9-rVis?tdMTTgBqqW zYkKu(tb&3LAu~krv2gg)r=R;lzpff=_-7#Ik(mJL{?zZp*BWX5w;?dG0EqaUI3U4J z4j7jl%)>idC%Bqr!_pQNmE&E@s=`|8!@ziSBH3AT@>xi2@zqbr{%bR;mzc zkp~ZqH-an$diTxxQXjTlzoVuT5UtT$W)!(Ema7JzA_&1$#lua_mz z$2sc&ahO+MzPtH>!T#j1CrMK2YuyJMBnLmg`Ui5Rvo?_335G=ToMQ?Vc2^touw1ulh~A7h=yEi7|C zd?2d#Re&}Emp#k$SRt>{8}~f-7LGrD{3x@6qT9HW8KbSI7vZ_zE$WX%JsG?Ag~6f5 z@gR=hFzfexHPYw7i?+yHSjPLW{s?C|pG2{Q21Bn|;YUlPkg@>6CZV!fdhJMWYKCu` zj|G_45PVX&(=U(}tF~q`QboO>$}l)EKzsA!pLxE{w9AB-lp8F5;9D(Dc-uJ` zKLus2W)r9}*!b93n&SzmU|tNaqjQK!bf`P!8Z=M?mWIzSi$@g(#R1VF52^*Xt%D=} z8QfkXf@PNvVjaUJz)REROgg?>h`-V`xzA)PG7&Sir7DKEI_uv9N)<)Yxo4*B_t~SW z=}2!vNLX04n`7M|t3qr=AXEE!34X{z6`u^jn|C2ZOiBJ1r@b_^v;v|@>B5~QK)LpM z{qp6!V^-;xyFhfW1N5aLN|}8lsPXXN;GhJjhH>DELG9aQ8HlxL?VEnjTL-cFs_pLc zhzHw!_4y%fTpAV3ueaQjb%jJEc~;A}r0n+RAHF5JpQsZ1b=)GvDNBeIxp|A&?xD>l zHd0QI_>uRfdJwahRZ~MCfyu!H38;U7b^xD(KbN3j?}Mbw3cKl=(?39FVp$F9%^OC9 zi_Ha#Tc-FPd4UM-h}gUgkzc}H&A;ca#qrn*_es@@Wfomb3Dv2yox*Pk$`ZTU%DbjG zrdve9Vxoy4T5NVhJL&2UhY}5d+It{oXT@_0Dlf(aNOEdef z^XVE-rREpK^O(CowyFxk7{&LlT`%19Ox~7>Ffx755nzUsz5aS5Y}S)ccL$jb7wT$? z;WMSBDo<>~L-p}|rAFAO%B8v6!d#~>UZKIAw+VX(W5gaGKt2z-&0p0xrChpoyN}`7 z2wB&#Yi&+(v7@fN{g&>acoh&&9s^Zq1AkE5tMd&ZsXZ_*2wl(v8ynHp8?5O{5q<;B z>lGLbf#ZG}G5&vHcrxs5Q+D4!(|KUCu!<$W5MF4T7t8%=clbg+GNpq_qy41DqFp5- zbYq~N`28bMhOpQ@HTNz&Sz?%Ek==yLr3N~&OSLvQtH2Qm5$WX$o3Fr0hCpbWn=cB+ z2^dKXxFjo}4-5T$*MJ@ze%Q3#SZ+rz<8y+HH}N|K`Z#{TI!QnV&Mfg)64ClAqGmJc zphM!gr83XF|96s8Om++H6}Q=WQds9o4Q>0?N+vIW95{XpW!}w<;mT}wWE|9jTuk7T zH(?UL2h~56g$9tJEF|n~sN5DjqVk4o-W{ungr1!JY$fivdl!lYlYlJ4s zm+8syp}TJYG>|~hLvI9-1viUaW zqiTKIV_^g6_5!O0zyAPQ2X~cJkgp3sq%u!q|!`;fPFBmPI*s#cLk%}>0?V<3p(I%s^kJ8Ly)K$Tfu zel!RE;#O8l;lC`D&Y9w~XL9Gsw;T7q$g9d#J;?I6g!%??`dL^%qEp^M{PGmGYQKT0 zba8(Gaq20reddW^H%O^Q1yfrzb8t?rzP<ph8~<#8mVB- z&iz60H$Z^D$-q$Aim%LryN{1+s_)4xO^?(V4*Li=6cj|}tr;rh2e8;r8v($7Wx{xQ zU-rQbQqYdV4;>pf*^E{dICTXrh;5=&yNNj9HwgoB`?|=Wf!8=L$IR>V4TD}@wPSJi~&90FSv)NzO4+X4e*Cn2m010NO|rprKP3mVLZlm zq%;67e4Vh$D6L|XgaY~q_BD`9sT~=Wet%*(-YxXUth?iMr7t+**T*YUI5?c9F(th1 zIm9ne_<5pZeKGh;;!OZUfxa8~m9l3dYmTNbQD5xv-2g zN7n0iLF+Cmpv>?yxa~Wyxu`#)k@~=@0%CrU*=MH_$itc0&s>LeMs&5c!>rJ@H`7sZ z=jT4FgW0_-vG6607Zj0zo(x`tQk8JTgk9S}d-=k%&Uq1#@RTk@mPxK%oDb5lq6&tZ z<<;Y*HyP#9ZLRmv=5EOs*^gnd;K5&Et|xnepVdcOwVXI}zEllXYGmc5SttCjTxZuF zPkYOH05{Q|`m(sDutgWwc=5~D----Rt`G_s`GzaeF;B5o+SvNmcyOTJZ=8ajF(y9@%R$ctEISC7y?eko|6+E zgu<_K-uDT+r-)t+vY_Fl*iijQ+=ma$CJLCT8%jH~UYMwxWRJ0Ztp zdYwb1dOw%N?RaONI_BjSSRpf52csrm*EEq(A`jz$20U^zEKak^l_Zr`222q&q`nLn z&sqY)Jh~pVCL9uszxT!B^7T0o+3G*GQ4Ja#wE`XPi9Tq_?E-Bi_}!yh;&Z{bx5Sse zyaXE1z!RIzL>et9@|8Xw(1V`Se3B5snoRpL5BO?5(D)Pt*;2^Y zcNn2+Y-~J+lmGMkS@D*{zR^>#R^qS{W079G%4|8JGN1d-3+~6&)fr`BZFW&ThOQBS z-nz<%^hNEGO9pI}@`VzSKCcIY-cop2*wvZt-+C|=Bcmzq6u+}oAnqOwEvNX)lvGun zUBwA(m$x~ed^-V1QU0r%<&|73kq(?*kFy<#J(nl>cIA*i?MC?8*KEa)LUYma^qV`b zoh8%4)Hx%7?feRikb7>097|^;rYs*Ca|hiTe-24W@6efP(aabT5s?F5;JVs=kYsLx zS=I1AbC#EXyF27ru-*ZrqxBCIEEl5b4?#N@SNL8xbi5iuI*z!IVD}W-i@yC>$Y@oR zNXG7dZ`5C5zxinkLjhXk45r-tKmt%Hbq&g-j5FsKrX{Jq0&As)AjU5j7&P`}Ljc}7X!fgd1HjKu zfRj((3`z@)6(kX!@_&VpY(W`1Yb z-nnE+VKJoCLemLcpz_{k-LUv3f*yh%sT=x|lgZ#R_AZsMYLMmMJA#XZm!tlR_5 z!jtM?+q+52c~UzK1VM$*D)-`x_0`l0A<#082U&U%duW11{j!P-5H@PV4^RL+{JCgK zVWqg4LxOiY*Y=IfmVq?eYbsA+w#b&aVXOmiPAgB0#OSBBP-(@%$H9MSwbELmhh?+1 z*K>tA5HVclA;AM_{2Xk^bS#3K1J*06%RMAu$cX*MCPV&jr<`phnE8Lk&pv9OW7;ra|_4pNX6@0XsNvswWFEbCJa~!9cp%e18BCPqNUYe33N2p!4?W6 zDkD3)GfIj1R4@HubZGs}OAsT@Zg-?Sb|jxAoPoa?;>0-O5{4-;&z2n1Jp=D~FMW*e zKP9V9#@s@z^Yre^2;ayXIr06Q4s1BU4r47r^!%CTc)r~PxF3F0!EWnHmzi8N%}q^b zeR}uRl*T@-SCx1juBqDC*pRAQu6C>!*89b?==N3>@Z$88U!ek3R-ab+Ajlthpr1!P zGM<%dVH5x!EdYq@JOPRJ^F>2uNG4T$K#KrDCGWd_ZO1`=gRiL<_7(7Opzo-Cd9C;GVc-dp7)Px@D{H~QZ$Hr_ zp=V$WTs>DsU}NwfY_US3uHLwtdJisJw6f&lF~yWT0|}rTgh%CT zv1juY%e26!H8kW~nwz@@5^-7e;jm$Em;xJTqF*7nY(Vup0V#VKbC4~2xcG3l`jvP6 z`61r=SCU_5QqU+};OZ=NH+`phgg187S+Uu~=9g)5-tT5XNZ2@gIgRGP!+le@!SB~I@2U^33RD`lf4WUS4fW0?#X05k1^mKna4T}F2L|t>W z`V)1xcB`Ng0eE2W!If6?5l4y6lH_|gNWHVpUq^BVY?-?81SE?%&NYA7_F0w`23m!$ zC}e987XxgCfd0z@7LL=uBXJIEwPng{S}<|kUGCXTwcPc?SALB9)>wE4swu(!OAnrZ zyV{6suz1$fllVNv-$!;)uiG|jhB4~GAzj2Bp7?AoC(yOssap-+kGu)N$iQ+n9@>a7 z2WjlJn(4niQ;+rb^O0q0jRL3+dp8Lvj~H)}o89*@-Z&0S&Zv(QjOtB6gtAfs;fnySRkDAov`(~sT=q${?DNzzP ze2#}aH!eHyuIs_o@gFR`kH8=a*k02JnX?toEFtjd6w_tagBt2|!QOJ_g-h@*0~-u23ll+m`UyJu>MO?$rF z+5?)6x&_=+keW#bv?rdW+82DjqweLn()Zomk9QIz!keEu(iB4rKy%TMTbN<0MdYg8>&0hJTRz`Sf|L6LX&Iky>bU-D<(Y zyZN-trc;(sF!dvJgUlNUjldEN+Vl`n60k6mU1sswVF7akzT(7;p)KS%IIpp>If5Hq zQ34l^aWiP>9a zA^hwe??SA4PMHn}{;)M7*Kc_op;6fdo@p8Wy4pg6HAe9=R!CPz+^9`5wy{eaIJ=u8 z@~GWt{^*!|j^j;~`dtQo$cGSDJ3~q6&SNdp|2*CHdS8&osX*Ym@O;||)zrF{{##tIuc7jkd^zmb+$AmdC*Hg-Z|iJ%*uLU*Yfc!7v48aLNPj^h||A6bBTC^fMM^uU1MIso&+lKI@f112b(L{PiF+Ojq4s+BW}9o(4!{$^9BF-z#mDO%r!CAX98-cy_#v9mm}lSKwd6PX_A3ZCn5t&Z_LgVpljofO5}B zd)A?*Nvt!;-7xp@dh}i-w|&RlY=JYi-im>1-1zuz>nZU6x^I%N01-^C>rMq zV-?^snT?bu5AjSs_5#9V(#!=9m^%Z)!VVbsV6j1ZyVx#Idv8nE7ZhxTC7&nYd_=2Y z)~5vwK%BKXwldM(PvX=$*(@c8F5`lO0PZnn=%5PiXT&2gHdvPb$^WmzTHIm>n4HS^7{K0#jfI{l7!$Mos_Sb2WT;Akm6);xO z3z%7)qx7Nha#zCVYG8mZ<>loSRx2a|mQvdZzs0>yLH`k}Na}Po>>OTd(CIcgIjd$2 z{Dt`ex^$IMxhDVW=@XsL#1TzK`wA*r7S9Ic?STL`{H0^zGA!qBgbxI%&2^)o|9Jrd zVlrc#!ht9Bva^#D;#{{6Odx?7N31ftlb;5ri$eDN^(fY1yJvny;4sQ59IgJA6D6U-Xf+OiYlJXG)OBPI;NRjVdX>Drsg6Epbc{;V;^o z!<1Ac-}!D)mPB@sF-dyQO)>W>OB^4b&n6VJ+icJqrlT2Gk1;;yfmZBEssok1PDaNi z&dWW^cWzzY`o;@f0T&G4=dC!aOjhnfsoz047JEg&Q0Hr~a-(R+e)NU^HHi=G!uRNV z;@Rn1bc7uxmkX}YQbT22v=1*IrKdl1KJRD2^v90b9$BjJ0tt>fRIw5W@fQPKH%m0z zM@L5)+&9Nj>@HL_+C1$a$QbOsmnCOBb^7GSHK~rBA+ehPcrf1Ie9GdA^pal@D|+3P zzhBU57Dhi}nav|KX%HPAR)`1@iSIOG;^P8PctBbEL9cW2{x=Oy$D4f1-@qsp6q?5Mpmf1}!BA5mJ`@c`Nf1DURH-Q+L)nZ};WXJ;r86(#^(k;SJFmmfBNsd4 zP3k;$UmR^Nh2FVyr;P@zh4Bw062>60D&%-N0gn`Ilv0m7Jm|Z}o*)|4|LfgC`cJ2A z!4}7wDE6f0=r?ciNoDYbePC6e2cHLGUZgbNy279POaNP+m$yEg!30KS!N&rP-_9EY z`fnwB%G2{xV6Z{bZF{=5pz%9X6o+cuGhe_ei+aqU++0V-AaVleCN2qlp!XT(6s>i3zm|AkrtFG!Fb=;Nx z2D+H9QqzKNz-y$wwr{WZ^!|2wdb;SHVhGL?yMXhEBqen7jyQMnS4h1aDKL7ea#AZH zDakTcWZL8(6rP?G3(M@g1t#Hay0@na%K>dP$&+nv9P3L8K*nKaD+H8@x+HJg+1qf0ZRZEblDZhgxyw3eEMU zUUAMb+XpT;prv3%?*RH|nS`&wn6kf2N-t-DMSd=vk9HAwNiS@KJG&R}<3eH#$nYtp zeZ@@S$Ht&>eAMQ-oOB%3R}4W2QUH?4^u#VSeh+Oph&N}zHA*w*$9%*TnG4=wnv3C>EHzT=rHZ=BeYpMY^VZfD1rg{v^+mTm!Plo8`!y=~$MxdF6YJHbmQRnLB>(EwPZ5n$OyXGW zq{|z|Y7QCVe~1;y8TIZ`1Jh{6v;=ZS7LUCNDYf0*k7citlgoT5(*ic}MW-=vma9eLT_LSbvm6a7m(3hP~YQzr=?sB=SJpF_WQVIkS{l0YXJ(fMK z{t6RO_~g6zdcoh{)0GR_68N~)U<>(fUz$*+Z1@4`pPQ4rpaEXTP&1LfPUEAJfsR<- z`f6aOnwm!U&1!=@SLf@!49?F4XJ;&|n0A&qpQXUgf>}G0n0NS3WKytpmCx$fT^LE* zy?e!JA$%O(X}ns9&QpnJxX^}KG(03LeR0fB(OG=W@?({nf&`nO?`fJCa=Nv2Y(ua~ zfe=#4ekKq^jt9Z#lK` z2rv_(H>mO0b=~&e_THwr2~1w*QN#E}sb0zGg5J=jJMyi8r|4nzEe`L{F#qIFqOpO` z47?;^IuUtmu9eU1y;F-`FNlQGZ(YiG|Dk6Tuif;KLwYJ0X7o`}QflqVINZ|H0VDO6 zBhtr{``^dwF*`xBQU%&PwNeLdG{TPc){Ily0`vDuYamOiqsR&T zqQB3+pV)mO8F?R0;B09@l_w?W!l+`-7VUVa)9YA8Ah#-EY`(T$T2mj`3aV@SR3O$X zXW!KS?Y9%wE)-G+8ZX~l4FbkV03km}UWU}8HWAJt zRp;v5Ygd`!JET1m+ZE1TW@2Cy78NrE^`h&-qyJ2@*iuC?nWp{f|NEF?R4{yAJM2SR zH@@)CpHG5&dU`^gDxZmmq|3Q&j^Fp%M2%cH2pblIfeWmGK>XX^3$^k6{rw?eY}!x6 ziuDI?V)q>zPNINxaKGiA9;95Xc8}lNxzf+IG4x<*VTu3ZAf#lWO<-tQ-R$RGZYFw@ zMAx6b03%g5Da7hqR|djApb6M|Dz(cDNLU-dZ!!6)2XnIh^ncp>%7-YMsP799P(r0e zKoIGckdly)27z6=6j%`HlwL&|NkNwGSX87X7F0x}Lr`f2i6tdhV&R?jex85f{q*c7 z@x$(1bIqJNzc^?7KVINH@m}mrm*(A;V2W1{1ORjB^tj{fSP;+wAGixO!oi}CEn4Y| zh=@?4yn23BdtQ+W(ue>HuME3lend?Z2sWZjz3sH6jk}}l8(opdCXU9kWls6;R`;g- zm#z9lfBxuwH$egw)J$Thk6ONC-1~s%NF9U0l$M0LFXyaJ)GcS`pPekP@$&LUB<@K_ z^epeT0`u}d-e*W5K-`Q&FQYz`7@E8Q;{%)KU)q!n)6Vm=Q}=Rjlg zQ7L`}w#7-(VGr22(Yr66FB0+sA*AI0{zT)44YmsmRbBVQJY!GX>pF&pG@d#2hMWgS$_0*583kcOXA2EzLj5k zWtPWeyWKC9JR4PDP*w@uD`pSHfkq#=w~aqOwBdj$FO@~)-Os0I6l|0T4MPUyDi99~ zu;Pa!>=k`;DXDGFmkUMYI)Yn$1D1YMeLJfdQsZI@G`HgWc3LUK86d4x1mn=UP;y?i z5IJ`9SCO0{ticx`alAE8+66Z4c|yh~7eb zErG4jWXd7ukD@^#O$rfJvq$}EN41Vv^n@RLzeeiOw`nFmAxR{PUqR~0-rkXH!ol^?H;W>Plz{ov(=l)o&Pf4(2(8*OXWTS2}9SyFR| z#&&$aZcA&cy4vYJs7Lw1D3^D+4OiLBlUT&(dmb9T2a@D-H+??`TuarRGX-JXMcO?@ z>Gv%+|C(5#4FPRMI#b7e2~>Z_IZ6uZ`v{d(;>#&7=!MPe$YnTw6;si&O{*A={{^+LG2(wzdEh27nC zWX4`&N<(9-!!r98C%yT?<40r?Rl6a(L^G0e%2yMfKgXNi%AU#nV<<=Y<{qDG^UtT0 zvrKXqA?08oMsV=WHJq@_%Wan2aB8@``{piMG*77Rq}x?j?Ji~7=#*hbZ?M@G{x#wj zjgXk6ZYwV@L_-7DQ&%m(5KAQRv!<aQlc zKu`PL4OXMoIxezl$WK}9bg%z-q%--a-Bn-rz-e-oUilhbirzpX`SWr#7!u?xwiP;M zYmHuwP*qh`0>R)sf`?$cQBpdd7@B33%NU-lfyf}O54noSpGRTKqvTzFC*?7XTLQ69 ztdEStKdT7|Vh!Zw;VCH}6)SW6ZQ#;xKs87cQ717nO5kN*p=u@_MFuqL0Kzd-fUj>o zoOZe&I8lDr=<@7hhikZ=pc=-vEF9y6`MA7DWKMO~kQUs-Wc(m^Ebv+;F^Texn)7m=kkS<1NwIqEfO=rB+|NUj~n*s?StdQ^N(Kf z*r!wL_-n6MwHH~Vbe9ARA84o4OS|NJ~si90ig0Ox70&b1}fYkSylNc35=RnuY(V zJI=C~(dO6WkbKM}qF_5 zF8mLx1YXsV7IYCtE|ax}uW?z*1JfO$;Xm<*WXd&ifwoCRA|=mNh~%(nEJQ;-i9j_G zN2cF;s8u3xLm8sc(i-jTH=^MkKKD>|=s_=-<|aS|1O&}xQT}KR*G3zSKFyE2%jXAxOJY9o>5nj`#RF3`~2mf(4Z~0E?{fc1Z=fMl6M_B1ZDkV+j`tR19+-9zBrZW(aQ)P@KQ=aY_dA)HErJ+8Q%W6 z^2w`TPX?Jmc&87VKduhNt~_3mu}DWzE7tUxl-dioFI8EcbqY{;p^x7BxIcP7Jj%nc z#$Sc5UwvBgK19v%#R-`pXI5&u15v>>c*Dz%lf}7>wY90dsI`*Ln-a`duDn|S{ZraO za08hRAwbuV>@6SuRxfS3k&%~Zv~fOp?fYv> z+`WRg0OT5V2IrrypArWSJhItMtEs7}Zu)p0wuO0RUbkdl9kM7G5Shqb*)Jsgb$sLl zq}=zg^%W=bMeI8Hyj&QAT!M*jcjrO<1M?LXc}fq?cuKm|c(l8=Jx#OlJxwz^{$_qs zj$;isq8~}Sra>5xMtoSR?QGH-?<3aTaJdH`rCxCOKieSCLqRKM!!mm}pdMj9Te&4`Wk=;e&6Lu8>!y9uX;;Z8&~4`h|t*YAG19{_KEHu zVPei?62@B!dcs9utmE717A2YwbV({qu8%)_D zO`-Yd3ttEztgFNu49bGde?C%%%ZZ64qZ4nTbniw*M|T7Q%4UvLj#2VOhBkN5o&Qqd zlBmUuP%DWU=~6D~G_y+zCf8bby(|a*BptO?QZ4I@emA78$|Tl-d)Rq(-$>0M>&xmz zYPu{Hlz$+0<|as0Y30EG)eU@*Ay+Bd5IlzT&+|bG$b3pR5W}N^`%>A3N&$DnAf|^& z?aG5tNgkn~4{FWUCNNaT4>gkmU2R1cokhHp$jjq3Ps96#l>)}65GQRnwqkeG<_U8~ z)HB(OThtzpJGlX1`Ck4Dm#|<4DyN?b+Jg3vbF|=La>~3Eu0k^OU@;kSugY!;_~qit z%8d_GP6Mq3 z1Hy3uN_9Z3;7z@K)c{`F)P#!%n6KkZdByB4hjTq-p}&T>OjIuvJd%H!R?#Z$%lfcB z@Fn|%th)|&&N4r?we+a|6MGx5tsT`8&4jRi$ATVI0>h` zHE_p$ol6~BrZc}SE&=r8i6#JdbyT~blB$+-K*9u-bj-ZO3CQTB`7{qFDC zf_K$oUyZEm9_@1XsJ9EG{mePuKiXf@`S4N2SZtbI^U^m!Xrc-29Noaqb1Ph^;Em&C zW@E#wqJ~120QE^q$(v}eUkh1K1ROy|{H-h4e5045qrcq3+@~F!QY5l+(O>R;FL9`M z>6m__#&e&>7j%<6$Nu*{`R3Nz+VTB=00=NIV4FPh^!!stsG-J1C6YtPWoD#RcxJQBVDA9iM+1`%UdGD7nHJ5C1WE=r zb0bMg8OfaSsWR2`H*7o~)OfJ7Ff%ucJ~XD;$KlS72>CI*aN+&a(GW#;;No)T6OV?B z`Wz)XQu)H^ij|j^Iy`XQO!in_dsx17`mezvD&H3+JV~DAXmLCP#6?*y0ST|YFxGDl zwzK=Wrj?Z^5ywfOgbAn{zhoF&>zR3t|R_^xW+jVrBWqBk?wSSU_la z)25C990&ZSo4`gi7h=8wnmA;N5_ZOG`^7DchSlk}VK2!{mk2?QfUTOu8FktAb=l3K z&$4MXr`zZec0{9K;zIC8^+<&Fq$Hj4`G@9|jTn-oj}?O|ztx?d&E@r6vZe>gUF(JobDzdef98q1^{g71y!L3W zu<3KiYj7rD+N;}*>dMuu=nw%=!t?_0x!Xh0QBenxtfUVR;xaOizY>%e$G05AFs&y22NM=_bY+5mjei;Ye2R-e?vH(RR7`o zu!j%pWQYN6wdr7MJBMp+X?dDJYzW7$uC@e@6sgZSAAP2IhSnymW->5M;Peo_o)&*P zuo&e&f9;Q*QtS7C=_a^M4-p|uS^Zx8k$1t%0&@n&*D!e`o|zk!XV0P;1EXi`7h>=t z)>F20_98H#XFfoXGL6ba8^iJM-@ku3yH_SXGq?Br#+AG3B+gQ-<52yx+2v5b^21Tb zX_9L|r+w_(#utC+vKp`F2;X`}FURwb{~n5rN0;79KR3$F;Fci;*!@c9IvH=Yg#~D~ z=|ma%(LqAv$>S|YXJ_XvlvCbeLzx?W>`%-x7iB>y$YY0TVQ?$lsz=;fTNT452@C2+ zb3oNAp zJN8ON@oIH%;NaXPCl4U3>a*z=)Qv=OPoT>L3y&~&FSV@@4=yrs!8sFk%lpT68Si3< zNR!ij&Rk|ud~R9;X;HMAgHCd$g@FJqJ6hm>wUMXXVx6#bvJtXDKq2c8d$sm#vl#et`59lL#6mcGdxURQUJtaVZJwUn(c@ z3uZuVX|xBOf$}fm14lv-Jd56_G$CY0>IF6klC-$Lo)A9aGt@Nex@aB2--|JM2;)51 zS;OmKG@ax+A|sde94qShBCX8Zl!kfV2Y1v|S69=ENuIMyhl@d48o{K){$S3kl|Wl0 zX$6q52r{8B1LhuZVBEIBKr#KBO)2uu?oJ~Sq%1+uyu5cForZk>#(pwro4wMl+U@=9 zs<*T~Vv}u-uR2L##=*&WPk=YUovHFmaMkJzz_F{y``0huMS`YET)~7G))bQfvSz}W zkX6IXm4FuSrB&_&StcZ`RoAIx^gU2RJA=IfM5LJJ(2fx6z;~`kFUX#arRu8Hro2qY zFU$*4a3Bsm>D4bq#Zx_bdtNSC#KDO`Ccxp}x=8+IiOyOPq9;eWSsQ0r&c_#h1+%WP z1eMB%(zLb=Z;7ZNF1lbK%Y>^mW`2d*<@Ms zX@~ZR_&H9_`RNh1JRNT9gD>vPi(&8DYfa7PIt+FuYBkkg^QZNM+_p12pKy*g`C7fH z-Q5-uZz3)mp0Dj|&cx1oU}*JiG7^8`Ubz8{U-7>>TTki+N+R}#g~lB8&J=r`LdI== zVa}@UZeHgDNpt9m7?Mk~t1?20K>pl@kIN+fub^JvHX_k}@~-c|Ix96tUv*s=TUjdO zOW)(bGx{>Vl-KrEkQIC@V9@O%y@{ZX`l2D6J9{ANBFTAuxbX-A2yT#_2;n7BPB#D5 zncrzD;bICN>#b?4RZzs91R7$8(n|$R8LRl>P!na_+uPPes;YLt7ob+eN5S?nM-0dq zk$u20h%_g}KSr{ATywD0SHDW<7g1EphK_#yJki%C90N7nD@5p0P7`Z=9sA(er+jf? zzJTr5Jx57ZSXuv7>?X(J3kopH5K8Ycbuo^#9%z@6vgB9i zhWQX>AYQa;0g0T29CCB~(`9;sQBz22aB*thnt8C+JDVY%Vi1bpRA{ah!scD}fUkv5 zHvk`XG)WZUQVsAlWX0fVURR)-D|YQ)%Z;D8U7Ct}AQbxg5SwT-0W;Zqu2TXhecJ=3 zgF%3YD+29e!5B`)XvCAIrGAMh10uRupa413!_x^I%Kz+v$wdEz5G`wWpGS$rHBRVW zWx=r8yMH)qYXhwM(X^z&7#!{##RUk+fq`4iLZoUXRw684$BC4$5T+#e!`BYqeDDQo zl2b|ELlFWq6SaH(FVxT)NLP74A2mkCRZ+jyAjYQs_16)# z_z6vtAn*R86(M94NLLGlsdK#!tyjH)VAd@Jjh#IQA=@*(oM7+j+m25@YAJXK6z8gAH~HWSZ_wY#DK7M=dPplD7zC%J zI=BctJqK~$yV3@m*7t+v@hRLx zrSIr{z$|SFX^ZT@r8wajPB1e$1iA-Mq*qfb5ucea$#5J#X9T?6We~9@ZMY7o{n&sE z-aYa|@p`zNq1?&tWJ;{&U?6trB^TP}tg{UVtR>_6qp2?pY0=6-6i+~~B71}c%a@gy zFf$%M`_HDiR@-^8T4UFKmyzSESaI!ssMtLOhKhlwD$H(qI$-DUgyaLU>ipZk63N%N zLY^>7f(?LF1$!Cd9Btm~2Vs|8)1LbG=jaWrB~Ck!STocH#<-|cTj8Vf3NU0<-Jd`E zlT%Scp3lq*7{E`zw*uh=qT$ck%NxjI|7*7*llTeeu`?u|;@fXdiXMY5hDX1oV$S}Gm>NeF9z84{-Lb?`_2WSu_CinWtLGMm3X~U_dpaG zraSj;>=pL70_Aprj)68VSV4JIqJf^8Q3kmLaj|dELm$ENf*9(3CV1689u#G3ul8dp zCR~FN%ZkBSr5;Of%Eaif1L>vGrm0!`-qMbB6XXO+Mk~F?L8chOKFIh)1;J}jc6VcIs1@U!+w`K?&M**A#MG?dnJDN6PzID?>%A+{^o_;Yjkuy} zGAha*bz@?qXX?X5kl7UU96si z_1QZDnu(g~52F?0m&8{L z(4?Pr!rQs@tCl(hs#ae=`Ka=A1jS0wzK5^(Gwi037@KucDBtMv>W{qnj}nCtG(9(8 zas`^?Jh`Y4T0}1w=O9kU@9+RSnC=q(VaD3p8W$Byp@l$jB|}dbAcC8piUn-!(J962k<|RG=`+l%lRLHW?^|qa} zCOF!`CM_-mD;}z61po9r&vP91%5d3FZ}wI1eY8lE;(Gq}AAK5)JTy2ze+{P6dwaRP z{K8Ilp#oD{gf2jHg%Q!j5`;q^^$5}f35vhaRZa7AY|2%?D#)N{vA^I!%a^LST*e(Xhs%^KnKA>%ts~nB15cq0)--Hl_IDS#GzCWZ?zEN5saqd6uPKK&! zg_NCU<6W!ylti8CV59%Lc}@9?z*QQRlzt214vmuY9V}9%%=P2Mi~5})Rv36FWx~bC==h-Zl-p(l}Ej* zqo4@wzPh%y;Bw6m#Vj4H+~UayzJds_-Lj1TEps2Spiy4jyF`0MKJnu**mK{C<{b9P zrJ(~JjNS#V$X**qts%AX-YN9a<6GHy$F~KOmKJCV7sT{B5&xYJgw9LYbLQat02;Ckbyn2NQ#Z|vB~f^(9{z-E(FB2CE&eY*wL6PJ9vAG z19f+)r(Y(T$oc$?DDaQ{+dOuOWHz4L{xv(?l$cyqy(jwh9l*mhC<&WSYlg zri>|c(Kyz<$AC>l>=xgd^z0;$Q8RQ%&8u94hHXF$xJYQl+pl_twQcWnZ0+Ei&kC!< z4os`PGN4OIM0m(k|EHS|yhL9!9oHZ@`Oecb8hz|{1x*sYR%DRFIgd^cH_n%G>jTC< z5^U)DEZr4w?zoXfbt|^*7rIq5r7UDM&xoFU4$#fm_+%vX@qBUwAiO}f`Q1CZ_ z+F7wm1%xAFbYvt&>3F>^De!CdY<;nrkV}{IWuL(wp))tMoCK}~XCsbE(Cb(pn|tN2 zEuS5C}#?6BFf;~#qm;;8<9f7pJ(kZ`oV+0a4Lp<TvF~o_>@qw|gc&Mz@buJ_eWD1l#Td*x&kid!8M0 zhTl4S^E0Lry*`UMj$}U?j&5h~-agfyUw88lI(sv9tvug$imd$9zKj}n+tdNgX3~g@ z3w`6#`gJHb%-YfN%}fSr!Z*e1Xv;5A5MEuGP}#`#t$q!C%kuCv-3xvzVxn;olRq)^ zs3W>}$fy~gg?r;t*RcJ)@vQPGY&GV}5=~@a%JlSH;Md8^R)u9(1OGlQffPGcIhZ14Iv?R zBfv0%-1DAhC3LbCcl!{(+@W6j_wa6jM}A&d*|##bkyh=RmsH=2$+vmLhYd4QHbaZN z3#QOMn{vz3Wyr$o+&Mne>-Y5&ZymCz(>xlGMEd9hbJ&VlvZuzBO|H+ZY=CDqsMCf@ z$N-z|`K&fqRf+MXW%9Kr4Vd8pUH8Br?Cf4+Z~EeUQwLKM7g_8+%k+H zs;4U^VMdyCcCkXNk*5+J^>81|S^e--EC!RM-@Ffyql673L)?RL5~o}0hQ6l2MhOLr z+`Eo7wj-*@cuK?3k0dQfNe$~KK?VZw2Wj+(25vf{-M_0~I#7#w)7%D*q=QHOu$H=- zJ--(vJEyjAX+j^)6wef*_aAb6xEPVXzw(%sr^Fm?QWTn}fAnHqOy#Mz%XEhk0y#Vh7F>*p?`Q0hF zx(Y?bt{nWI9|xlDqP_x2z8_{jYu12W&GS6#BKc;opMO>2s93~9Ks&h9A4*O_Fy2K^upl08GRlv3prmNpbFKI^ieII)mQuN#1eXzSM=ItIJlA3 zgHy1Dw@bDv>w15G7^s98D+(cpX&!N|2$fN>7NAq3_}^YB0R{@O)=#$JaV)4Xe}l8R zrglAwEL_KeKiiPi=_lP_hv&d0p0FhUC4p9|6 zuRy=(b3~(oAj9MX4J)TiNOw8d7teZlmA{^ny^rx-m(PsVjNTQW4)i2)@b%rx;N4|1GAr!1!BW@;{F0{%K>V`$fF3|fnZajF^MoxbN z!VfNtOoJ;@7||{W#KaVP8k5GJ{42p*DUR0M*a5gGax>@H?EPElXL70{sYxY_f{i$t zPfOH7zwkWq&kDWgC&QKD(6IQZ=S^}5Tf9?K+)Ov1uGl_rBI~xMNK@7!2OdvkM|@P_ zI;vtddlp=4XZ_?OF|t!T2q0;xWma52H2GA2=5f##i3TzBybJ6GA!O>s5L2n6#gz8; z;l+jgS$k0jo&(^m3#zrb8~nM{JI(@*>5|2nz58;HJox6lJ8+3Ydq^zcAs6LnB#olu7sIde-J;v30aDxKGihd$M zkH5%VrOy3(O9u?>22A$}>A!*nNa%9>Z}Orbe28s1r;*U3<-7~Qaxe4u1L?3sOx*d~ z4V{#m7N9gUPt1_$eIW`};d0BDHTnaJydpX3;o6HsBd{D1giAmezZ5ay9jcu7CJx*6 z$F-Z3zWM0$XB0-pR585Jb5x(3MWgT65hI2 za52_yC-Oeg`};Ly=aWUySt~`{NZ)&Kola+AxFd`inE^Nrh%qx@vwG>&H7!=A?m_oE zA>onxZ{GZXWds;#1~giYFOJAcZ(%=#9vj04Al2p zWED)+{S$QdR%~K9-J8|~vcG(d!o`g18ndg9ND)l%%_s{Ju8|wc;O>}}%g9ffeVibR z`;Qhevy0b18@cwpWmf+C^45>F-?3hckacvdrp9l_hd8y3`~y%2U`hP!9JHIdM|>VY zs$;#T$dc-*8!D}&!)Bmm7Q-cFo$DQBke*%5SxrpX;@csK6osyYbwT<2tK$&>LoD@< z=Egr-hELW22Vw9>;D`5F&e8D&ue*U>=;>ex9*iYA`7a@KkjC4b#w!TyP}59Etfj$9 z)^l&y%k$3q+^|eyN>3R~{Vm~`5=S}=-|rxfic~GO6u9>UXq#hNA=w}rBJai(ePTa= z>}#@e!|5mw8TdZ z=#YwxC>p|*UXsb;1TswP1nh}=3OK_}9D3qRWaratj%)rGaXg%jhf`s~jFExo6Ffu- zfQcHjHYGDVNK9#Bn20sbXYg5yB{F8c!(XTKDhHP!VI69KH=X4?A{bVu1@Exj@<8+8 zHdj(}ZfwDlq|>`{xtRrYW3q> zI!F&Ot7y2DC4RtpX|qR1jkmo;S2s#7el%TJX8;!vjbEv8VOG~4_gQR$Pg-E5AR{4gk2Kut{BLaja@i-;_#4CzxOr?AR9a?JsRWa-FxwqR|byyjaTOL#c(zMzaG2bhDO;^nw<~S$@VGGd{ zKw8Uu`H4xGxdGhj^N;tx{N<;P$Hbc+BuJ^I_@#Y&9` zag;}7+y$R9&eXqHRU|^Qlio}vn-pCFu6Ke_@i$4cN$uZF=>{z{MU8s9*&}_et z@4)132kjGYVbwDh~}db*X9MM(74j!-U+B(t4Ox zO*PwojpI>Gv1?sJb^b=E%T&*=2RUm_6)Z_U?%%fdoF~^t8uwK(?KauDLAS#{vAEL0 zL(SAOVl;2#?+qxWkQ^Ir%(-)V}sK&p2`ZF!#dbtKHbN- zedoWXiyO=hD-bxX{N8_cXm2-Hew`~29y{ENgTc3Ft9%Ca@MqX^);*MqVrOA$uEjP> zf7_cSDT|uqrRxj%LDn-*bunkQ$Dy#z8+Ch8Z?Y?1KW}|-mLp7Kb!f@vZCmm9OOd^cMI4+x-pYm5PsI5f?w|0*eUjxhb_@CJ>gIoK!*9vYxH@;=Z-3I2e&Dw| zI*!Sx0}sc{xgLk=Wh=T8am5v>HatqnTDa$yEhXV+EU(MjO{;D=V;}qF3F)Ip;pC{& z0TX)BOD=WBQ%-e}_;+YDZWex`xSHdGC^J4z&OX8*PEjbILG0Mp zW*Kg%(*9Y{*38%`8nINCc%6B9xI3=Q|5&iC?Cq*(?l%F;NUR=9HZw6R_+cMuRYYf-g%J0FWq%kbGWAXxvZ0pAH? zJp?Hi5$Mu$kB=ScSX$~%KQhcLoK+k5#os%RTGDgqdT#@al`sWG8JQnS;^MUu_kgd1 z@X?e30W6s#d#hu2E6(xQG6jphA)_0=E=qQ|y06s83TzgABvF981Rvxxdq5v*Cda0R z!PJH%>z28FU4E#2aS+eH$oAhLpj6?BDluZ^Y(l-kEjT_iMwj)V2kW45XrRU=^0H4_ zQrh+3S3xDzAmVU;dgH`j0eS1pkrvjh_yZYyg!&zp&g8Ia&sB<8k{NLyQ hX5jzN3|w75BhHD*Y2YArBM2&xmb$*$8x^~!{|^Edibwze literal 0 HcmV?d00001 From 0eb421c2bddb68a640dc28e8d69bfe1e79b9ef5c Mon Sep 17 00:00:00 2001 From: Chris Abraham Date: Thu, 5 Dec 2024 20:03:22 -0800 Subject: [PATCH 2/4] Move blog post over to Community blog Signed-off-by: Chris Abraham --- _community_blog/vllm-joins-pytorch.md | 10 ++++++++++ _posts/2024-12-05-vllm-joins-pytorch.md | 2 ++ 2 files changed, 12 insertions(+) create mode 100644 _community_blog/vllm-joins-pytorch.md diff --git a/_community_blog/vllm-joins-pytorch.md b/_community_blog/vllm-joins-pytorch.md new file mode 100644 index 000000000000..f55744ff973d --- /dev/null +++ b/_community_blog/vllm-joins-pytorch.md @@ -0,0 +1,10 @@ +--- +title: "vLLM Joins PyTorch Ecosystem: Easy, Fast, and Cheap LLM Serving for Everyone" +author: vLLM Team +ext_url: /blog/vllm-joins-pytorch/ +date: Dec 5, 2024 +--- + +We’re thrilled to announce that the [vLLM project](https://github.com/vllm-project/vllm) has become a PyTorch ecosystem project, and joined the PyTorch ecosystem family! + +Running large language models (LLMs) is both resource-intensive and complex, especially as these models scale to hundreds of billions of parameters. That’s where vLLM comes in — a high-throughput, memory-efficient inference and serving engine designed for LLMs. diff --git a/_posts/2024-12-05-vllm-joins-pytorch.md b/_posts/2024-12-05-vllm-joins-pytorch.md index 8a8f34a4ced0..4005633b263c 100644 --- a/_posts/2024-12-05-vllm-joins-pytorch.md +++ b/_posts/2024-12-05-vllm-joins-pytorch.md @@ -1,6 +1,8 @@ --- layout: blog_detail title: "vLLM Joins PyTorch Ecosystem: Easy, Fast, and Cheap LLM Serving for Everyone" +author: vLLM Team +hidden: true --- ![vllm logo](/assets/images/vllm.png){:style="width:100%;display: block;max-width:400px; margin-left:auto; margin-right:auto;"} From 21d464900260bf8c9f9a1f4d313df689fbc0b382 Mon Sep 17 00:00:00 2001 From: Chris Abraham Date: Mon, 9 Dec 2024 08:51:44 -0800 Subject: [PATCH 3/4] update Signed-off-by: Chris Abraham --- ...s-pytorch.md => 2024-12-09-vllm-joins-pytorch.md} | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) rename _posts/{2024-12-05-vllm-joins-pytorch.md => 2024-12-09-vllm-joins-pytorch.md} (79%) diff --git a/_posts/2024-12-05-vllm-joins-pytorch.md b/_posts/2024-12-09-vllm-joins-pytorch.md similarity index 79% rename from _posts/2024-12-05-vllm-joins-pytorch.md rename to _posts/2024-12-09-vllm-joins-pytorch.md index 4005633b263c..a3440a13c368 100644 --- a/_posts/2024-12-05-vllm-joins-pytorch.md +++ b/_posts/2024-12-09-vllm-joins-pytorch.md @@ -11,17 +11,7 @@ We’re thrilled to announce that the [vLLM project](https://github.com/vllm-pro Running large language models (LLMs) is both resource-intensive and complex, especially as these models scale to hundreds of billions of parameters. That’s where vLLM comes in — a high-throughput, memory-efficient inference and serving engine designed for LLMs. -Originally built around the innovative [PagedAttention algorithm](https://arxiv.org/abs/2309.06180), vLLM has grown into a comprehensive, state-of-the-art inference engine. A thriving community is also continuously adding new features and optimizations to vLLM, including the following: - - - -* [Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve](https://www.usenix.org/conference/osdi24/presentation/agrawal) -* [Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving](https://arxiv.org/abs/2407.00079) -* [Llumnix: Dynamic Scheduling for Large Language Model Serving](https://arxiv.org/abs/2406.03243) -* [CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving](https://blog.vllm.ai/2024/07/25/lfai-perf.html#:~:text=CacheGen%3A%20KV%20Cache%20Compression%20and%20Streaming%20for%20Fast%20Large%20Language%20Model%20Serving) -* [vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention](https://blog.vllm.ai/2024/07/25/lfai-perf.html#:~:text=vAttention%3A%20Dynamic%20Memory%20Management%20for%20Serving%20LLMs%20without%20PagedAttention) -* [Andes: Defining and Enhancing Quality-of-Experience in LLM-Based Text Streaming Services](https://blog.vllm.ai/2024/07/25/lfai-perf.html#:~:text=Andes%3A%20Defining%20and%20Enhancing%20Quality%2Dof%2DExperience%20in%20LLM%2DBased%20Text%20Streaming%20Services) -* [SGLang: Efficient Execution of Structured Language Model Programs](https://blog.vllm.ai/2024/07/25/lfai-perf.html#:~:text=SGLang%3A%20Efficient%20Execution%20of%20Structured%20Language%20Model%20Programs) +Originally built around the innovative [PagedAttention algorithm](https://arxiv.org/abs/2309.06180), vLLM has grown into a comprehensive, state-of-the-art inference engine. A thriving community is also continuously adding new features and optimizations to vLLM, including pipeline parallelism, chunked prefill, speculative decoding, and disaggregated serving. Since its release, vLLM has garnered significant attention, achieving over 31,000 GitHub stars—a testament to its popularity and thriving community. This milestone marks an exciting chapter for vLLM as we continue to empower developers and researchers with cutting-edge tools for efficient and scalable AI deployment. Welcome to the next era of LLM inference! From 8977eb3d3680473f2960c0f727fe50cdffc8682e Mon Sep 17 00:00:00 2001 From: Chris Abraham Date: Mon, 9 Dec 2024 08:52:45 -0800 Subject: [PATCH 4/4] update date Signed-off-by: Chris Abraham --- _community_blog/vllm-joins-pytorch.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_community_blog/vllm-joins-pytorch.md b/_community_blog/vllm-joins-pytorch.md index f55744ff973d..fcdba719232c 100644 --- a/_community_blog/vllm-joins-pytorch.md +++ b/_community_blog/vllm-joins-pytorch.md @@ -2,7 +2,7 @@ title: "vLLM Joins PyTorch Ecosystem: Easy, Fast, and Cheap LLM Serving for Everyone" author: vLLM Team ext_url: /blog/vllm-joins-pytorch/ -date: Dec 5, 2024 +date: Dec 9, 2024 --- We’re thrilled to announce that the [vLLM project](https://github.com/vllm-project/vllm) has become a PyTorch ecosystem project, and joined the PyTorch ecosystem family!