{"id":4722,"date":"2024-04-18T22:49:14","date_gmt":"2024-04-18T22:49:14","guid":{"rendered":"https:\/\/game.intel.com\/?p=4722"},"modified":"2024-05-29T21:16:37","modified_gmt":"2024-05-29T21:16:37","slug":"wield-the-power-of-llms-on-intel-arc-gpus","status":"publish","type":"post","link":"https:\/\/game.intel.com\/tr\/stories\/wield-the-power-of-llms-on-intel-arc-gpus\/","title":{"rendered":"Intel\u00ae Arc\u2122 GPU'larda LLM'lerin G\u00fcc\u00fcn\u00fc Kullan\u0131n"},"content":{"rendered":"
\n

Intel\u00ae Arc\u2122 GPU'lar ile \u00c7e\u015fitli LLM'leri Yerel Olarak Kolayca \u00c7al\u0131\u015ft\u0131r\u0131n<\/h3>\n<\/blockquote>\n\n\n\n

\u00dcretken yapay zeka, i\u00e7erik olu\u015fturmada m\u00fcmk\u00fcn olan\u0131n manzaras\u0131n\u0131 de\u011fi\u015ftirdi. Bu teknoloji, daha \u00f6nce hayal edilmemi\u015f g\u00f6r\u00fcnt\u00fcler, videolar ve yaz\u0131lar sunma potansiyeline sahip. B\u00fcy\u00fck dil modelleri (LLM'ler), yapay zeka \u00e7a\u011f\u0131nda man\u015fetlere \u00e7\u0131karak herkesin \u015fark\u0131 s\u00f6zleri olu\u015fturmas\u0131na, karma\u015f\u0131k fizik sorular\u0131na yan\u0131t almas\u0131na veya bir slayt sunumu i\u00e7in taslak haz\u0131rlamas\u0131na olanak tan\u0131yor. Ve bu yapay zeka \u00f6zelliklerinin art\u0131k buluta ya da abonelik hizmetlerine ba\u011fl\u0131 olmas\u0131 gerekmiyor. \u00c7\u0131kt\u0131lar\u0131n\u0131 \u00f6zelle\u015ftirmek i\u00e7in model \u00fczerinde tam kontrole sahip oldu\u011funuz kendi bilgisayar\u0131n\u0131zda yerel olarak \u00e7al\u0131\u015fabilirler.<\/p>\n\n\n\n

Bu makalede, Intel\u00ae Arc\u2122 A770 16GB grafik kart\u0131na sahip bir bilgisayarda pop\u00fcler b\u00fcy\u00fck dil modellerini (LLM'ler) nas\u0131l kuraca\u011f\u0131n\u0131z\u0131 ve deneyece\u011finizi g\u00f6sterece\u011fiz. Bu e\u011fitimde Mistral-7B-Instruct LLM kullan\u0131lacak olsa da, ayn\u0131 ad\u0131mlar Phi2, Llama2 vb. gibi se\u00e7ti\u011finiz bir PyTorch LLM ile de kullan\u0131labilir. Ve evet, en yeni Llama3 modeli ile de!<\/p>\n\n\n\n

IPEX-LLM<\/h2>\n\n\n\n

Ayn\u0131 temel kurulumu kullanarak \u00e7e\u015fitli modelleri \u00e7al\u0131\u015ft\u0131rabilmemizin nedeni IPEX-LLM<\/a>PyTorch i\u00e7in bir LLM k\u00fct\u00fcphanesi. PyTorch'un \u00fczerine in\u015fa edilmi\u015ftir. PyTorch i\u00e7in Intel\u00ae Uzant\u0131s\u0131<\/a> ve Intel donan\u0131m\u0131 i\u00e7in en son performans optimizasyonlar\u0131 ile son teknoloji LLM optimizasyonlar\u0131 ve d\u00fc\u015f\u00fck bit (INT4\/FP4\/INT8\/FP8) a\u011f\u0131rl\u0131k s\u0131k\u0131\u015ft\u0131rmas\u0131 i\u00e7erir. IPEX-LLM, Intel donan\u0131m\u0131 i\u00e7in Xe<\/sup>-Geli\u015fmi\u015f performans i\u00e7in Arc A serisi grafik kartlar\u0131 gibi Intel ayr\u0131k GPU'larda XMX AI h\u0131zland\u0131rmas\u0131n\u0131 \u00e7eker. Linux s\u00fcr\u00fcm 2 i\u00e7in Windows Alt Sistemi, yerel Windows ortamlar\u0131 ve yerel Linux \u00fczerinde Intel Arc A serisi grafikleri destekler.<\/p>\n\n\n\n

Ve t\u00fcm bunlar yerel PyTorch oldu\u011fundan, PyTorch modellerini ve giri\u015f verilerini y\u00fcksek performansl\u0131 h\u0131zland\u0131rma ile Intel Arc GPU'da \u00e7al\u0131\u015ft\u0131rmak i\u00e7in kolayca de\u011fi\u015ftirebilirsiniz. Bu deney bir performans kar\u015f\u0131la\u015ft\u0131rmas\u0131 olmadan tamamlanamazd\u0131. Intel Arc i\u00e7in a\u015fa\u011f\u0131daki talimatlar\u0131 ve rakipler i\u00e7in yayg\u0131n olarak bulunan talimatlar\u0131 kullanarak, benzer bir fiyat segmentinde konumland\u0131r\u0131lm\u0131\u015f iki ayr\u0131 GPU'ya bakt\u0131k.<\/p>\n\n\n\n

\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n\n\n\n

\u00d6rne\u011fin, IPEX-LLM k\u00fct\u00fcphanesi ile Mistral 7B modelini \u00e7al\u0131\u015ft\u0131r\u0131rken, Arc A770 16GB grafik kart\u0131 saniyede 70 token (TPS) veya CUDA kullanan GeForce RTX 4060 8GB'den 70% daha fazla TPS i\u015fleyebilir. Bu ne anlama geliyor? Genel bir kural olarak 1 token bir kelimenin 0,75'ine e\u015fde\u011ferdir ve iyi bir kar\u015f\u0131la\u015ft\u0131rma ortalama insan okuma h\u0131z\u0131 saniyede 4 kelime<\/a> veya 5,3 TPS. Arc A770 16GB ekran kart\u0131, ortalama bir insan\u0131n okuyabilece\u011finden \u00e7ok daha h\u0131zl\u0131 kelime \u00fcretebilir!<\/p>\n\n\n\n

Dahili testlerimiz, Arc A770 16GB grafik kart\u0131n\u0131n bu yetene\u011fi ve RTX 4060'a k\u0131yasla geni\u015f bir model yelpazesinde rekabet\u00e7i veya lider performans sunabildi\u011fini ve Intel Arc grafiklerini yerel LLM y\u00fcr\u00fctme i\u00e7in m\u00fckemmel bir se\u00e7im haline getirdi\u011fini g\u00f6stermektedir.<\/p>\n\n\n\n

\u015eimdi Arc A serisi GPU'nuzda LLM'leri kullanmaya ba\u015flaman\u0131z i\u00e7in kurulum talimatlar\u0131na ge\u00e7elim.<\/p>\n\n\n\n

Kurulum Talimatlar\u0131<\/h2>\n\n\n\n

Ortam\u0131 ayarlamak i\u00e7in bu sayfaya da ba\u015fvurabiliriz: IPEX-LLM'yi Intel GPU ile Windows'a y\u00fckleyin - IPEX-LLM en son belgeleri<\/a><\/p>\n\n\n\n

1. Ayg\u0131t y\u00f6neticisinde t\u00fcmle\u015fik GPU'yu devre d\u0131\u015f\u0131 b\u0131rak\u0131n.<\/p>\n\n\n\n

2. \u0130ndirin ve y\u00fckleyin Anaconda<\/a>.<\/p>\n\n\n\n

3. Kurulum tamamland\u0131ktan sonra Ba\u015flat men\u00fcs\u00fcn\u00fc a\u00e7\u0131n, Anaconda Prompt'u aray\u0131n, y\u00f6netici olarak \u00e7al\u0131\u015ft\u0131r\u0131n ve a\u015fa\u011f\u0131daki komutlar\u0131 kullanarak sanal bir ortam olu\u015fturun. Her komutu ayr\u0131 ayr\u0131 girin:<\/p>\n\n\n\n

conda create -n llm python=3.10.6\n\nconda akti\u0307f llm\n\nconda install libuv\n\npip install dpcpp-cpp-rt==2024.0.2 mkl-dpcpp==2024.0.0 onednn==2024.0.0 gradio\n\npip install --pre --upgrade ipex-llm[xpu] --extra-index-url https:\/\/pytorch-extension.intel.com\/release-whl\/stable\/xpu\/us\/\n\npip install transformers==4.38.0<\/code><\/pre>\n\n\n\n
4. demo.py ad\u0131nda bir metin belgesi olu\u015fturun ve C:\\Users\\Your_Username\\Documents veya se\u00e7ti\u011finiz bir dizine kaydedin.<\/p>\n\n\n\n
5. Favori edit\u00f6r\u00fcn\u00fczle demo.py dosyas\u0131n\u0131 a\u00e7\u0131n ve a\u015fa\u011f\u0131daki kod \u00f6rne\u011fini i\u00e7ine kopyalay\u0131n:<\/p>\n\n\n\n
from transformers import AutoTokenizer\nfrom ipex_llm.transformers import AutoModelForCausalLM\nithal me\u015fale\nimport intel_extension_for_pytorch\n\ndevice = \"xpu\" # modelin y\u00fcklenece\u011fi cihaz\n\nmodel_id = \"mistralai\/Mistral-7B-Instruct-v0.2\" # huggingface model kimli\u011fi\n\ntokenizer = AutoTokenizer.from_pretrained(model_id)\nmodel = AutoModelForCausalLM.from_pretrained(model_id, load_in_4bit=True, torch_dtype=torch.float16)\nmodel = model.to(cihaz)\n\nmesajlar = [\n    {\"rol\": \"kullan\u0131c\u0131\", \"i\u00e7erik\": \"En sevdi\u011finiz \u00e7e\u015fni nedir?\"},\n    {\"rol\": \"asistan\", \"i\u00e7erik\": \"Taze limon suyunu s\u0131kmay\u0131 olduk\u00e7a severim. Mutfakta ne pi\u015firiyorsam ona do\u011fru miktarda lezzet kat\u0131yor!\"},\n    {\"rol\": \"kullan\u0131c\u0131\", \"i\u00e7erik\": \"Mayonez tarifleriniz var m\u0131?\"}\n]\n\nencodeds = tokenizer.apply_chat_template(messages, return_tensors=\"pt\")\n\nmodel_inputs = encodeds.to(device)\nmodel.to(cihaz)\n\ngenerated_ids = model.generate(model_inputs, max_new_tokens=1000, do_sample=True)\ndecoded = tokenizer.batch_decode(generated_ids)\nprint(decoded[0])<\/code><\/pre>\n\n\n\n
\u00d6rnek koddan olu\u015fturulan kod bu depoda<\/a>.<\/em><\/p>\n\n\n\n
6. demo.py dosyas\u0131n\u0131 kaydedin. Anaconda'da cd komutunu kullanarak demo.py dosyas\u0131n\u0131n bulundu\u011fu dizine gidin ve Anaconda komut isteminde a\u015fa\u011f\u0131daki komutu \u00e7al\u0131\u015ft\u0131r\u0131n:<\/p>\n\n\n\n
python demo.py<\/code><\/pre>\n\n\n\n
Art\u0131k mayonez yapmak i\u00e7in g\u00fczel bir tarif alabilirsiniz!<\/p>\n\n\n\n
\"\"<\/figure>\n\n\n\n
De\u011fi\u015fen Modeller<\/h2>\n\n\n\n
Yukar\u0131da kurdu\u011fumuz ayn\u0131 ortam\u0131 kullanarak, demo.py'de yukar\u0131daki Hugging Face model kimli\u011fini de\u011fi\u015ftirerek llama2-7B-chat-hf, llama3-8B-it, phi-2, gemma-7B-i ve stablelm2 gibi Hugging Face \u00fczerindeki di\u011fer pop\u00fcler modelleri deneyebilirsiniz.<\/p>\n\n\n\n
model_id = \"mistralai\/Mistral-7B-Instruct-v0.2\" # huggingface model kimli\u011fi\n\ni\u00e7in\n\nmodel_id = \"stabilityai\/stablelm-2-zephyr-1_6b\" # kucaklayan y\u00fcz model kimli\u011fi<\/code><\/pre>\n\n\n\n
Farkl\u0131 modeller farkl\u0131 bir transformers paketi s\u00fcr\u00fcm\u00fc gerektirebilir, demo.py'yi ba\u015flat\u0131rken hatalarla kar\u015f\u0131la\u015f\u0131rsan\u0131z, transformers'\u0131 y\u00fckseltmek \/ d\u00fc\u015f\u00fcrmek i\u00e7in a\u015fa\u011f\u0131daki ad\u0131mlar\u0131 izleyin:<\/p>\n\n\n\n
    \n
  1. Anaconda \u0130stemini A\u00e7\u0131n<\/li>\n\n\n\n
  2. conda akti\u0307f llm<\/li>\n\n\n\n
  3. pip install transformers==4.37.0<\/li>\n<\/ol>\n\n\n\n
    Do\u011frulanm\u0131\u015f transformat\u00f6r versiyonlar\u0131:<\/strong><\/p>\n\n\n\n
    Model Kimli\u011fi<\/th>Transformat\u00f6r paket versiyonlar\u0131<\/th><\/tr><\/thead>
    meta-llama\/Lama-2-7b-chat-hf<\/td>4.37.0<\/td><\/tr>
    meta-llama\/Meta-Llama-3-8B-Instruct<\/td>4.37.0<\/td><\/tr>
    stabilityai\/stablelm-2-zephyr-1_6b<\/td>4.38.0<\/td><\/tr>
    mistralai\/Mistral-7B-Instruct-v0.2<\/td>4.38.0<\/td><\/tr>
    microsoft\/phi-2<\/td>4.38.0<\/td><\/tr>
    google\/gemma-7b-it<\/td>4.38.1<\/td><\/tr>
    THUDM\/chatglm3-6b<\/td>4.38.0<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
    Bellek gereksinimleri modele ve \u00e7er\u00e7eveye g\u00f6re de\u011fi\u015febilir. IPEX-LLM ile \u00e7al\u0131\u015fan Intel Arc A750 8GB i\u00e7in Llama-2-7B-chat-hf, Mistral-7B-Instruct-v0.2, phi-2 veya chatglm3-6B kullanman\u0131z\u0131 \u00f6neririz.<\/p>\n\n\n\n
    Bir ChatBot WebUI Uygulama<\/h2>\n\n\n\n
    \u015eimdi web taray\u0131c\u0131n\u0131z\u0131 kullanarak daha iyi bir deneyim i\u00e7in bir Gradio chatbot webui uygulamaya ge\u00e7elim. LLM'lerle etkile\u015fimli bir sohbet botu uygulama hakk\u0131nda daha fazla bilgi i\u00e7in \u015fu adresi ziyaret edin https:\/\/www.gradio.app\/guides\/creating-a-chatbot-fast<\/a><\/p>\n\n\n\n
    1. Se\u00e7ti\u011finiz metin d\u00fczenleyicisinde chatbot_gradio.py ad\u0131nda bir belge olu\u015fturun.<\/p>\n\n\n\n
    2. A\u015fa\u011f\u0131daki kod par\u00e7ac\u0131\u011f\u0131n\u0131 kopyalay\u0131p chatbot_gradio.py dosyas\u0131na yap\u0131\u015ft\u0131r\u0131n:<\/p>\n\n\n\n
    gr olarak gradio i\u00e7e aktar\nithal me\u015fale\nimport intel_extension_for_pytorch\nfrom ipex_llm.transformers import AutoModelForCausalLM\nfrom transformers import AutoTokenizer, StoppingCriteria, StoppingCriteriaList, TextIteratorStreamer\nfrom threading import Thread\n\nmodel_id = \"mistralai\/Mistral-7B-Instruct-v0.2\"\n\ntokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)\nmodel = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True, optimize_model=True, load_in_4bit=True, torch_dtype=torch.float16)\nmodel = model.half()\nmodel = model.to(\"xpu\")\nclass StopOnTokens(StoppingCriteria):\n    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs) -> bool:\n        stop_ids = [29, 0]\n        for stop_id in stop_ids:\n            if input_ids[0][-1] == stop_id:\n                return True\n        return False\n\ndef predict(message, history):\n    stop = StopOnTokens()\n    history_format = []\n    for insan, asistan in ge\u00e7mi\u015f:\n        history_format.append({\"role\": \"user\", \"content\": human })\n        history_format.append({\"rol\": \"asistan\", \"i\u00e7erik\":asistan})\n    history_format.append({\"role\": \"user\", \"content\": message})\n\n    prompt = tokenizer.apply_chat_template(history_format, tokenize=False, add_generation_prompt=True)\n    model_inputs = tokenizer(prompt, return_tensors=\"pt\").to(\"xpu\")\n    streamer = TextIteratorStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)\n    generate_kwargs = dict(\n        model_inputs,\n        streamer=streamer,\n        max_new_tokens=300,\n        do_sample=True,\n        top_p=0,95,\n        top_k=20,\n        s\u0131cakl\u0131k=0,8,\n        num_beams=1,\n        pad_token_id=tokenizer.eos_token_id,\n        stopping_criteria=StoppingCriteriaList([stop])\n        )\n    t = Thread(target=model.generate, kwargs=generate_kwargs)\n    t.start()\n\n    partial_message = \"\"\n    for new_token in streamer:\n        if new_token != '<':\n            partial_message += new_token\n            yield partial_message\n\ngr.ChatInterface(predict).launch()<\/code><\/pre>\n\n\n\n
    3. Yeni bir anaconda komut istemi a\u00e7\u0131n ve a\u015fa\u011f\u0131daki komutlar\u0131 girin:<\/p>\n\n\n\n