kaggle运行报错RuntimeError: cutlassF: no kernel found to launch!
项目场景：项目场景：使用原始Llama3推理，到这里都是能行的
!pip install -q modelscope
import torch
from modelscope import snapshot_download, AutoModel, AutoTokenizer
import os
model_dir = snapshot_download('LLM-Research/Meta-Llama-3-8B-Instruct', cache_dir='/root/autodl-tmp', revision='master')
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
tokenizer = AutoTokenizer.from_pretrained(model_dir)
model = AutoModelForCausalLM.from_pretrained(
model_dir, torch_dtype="auto", device_map="auto"
)
题目描述
RuntimeError: cutlassF: no kernel found to launch!
messages = [
{
'role':'user',
'content':"""hello"""
}
]
input_ids = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)
outputs = model.generate(
input_ids=input_ids,
max_new_tokens=8192,
do_sample=True,
temperature=0.6,
top_p=0.9,
)
response = outputs:]
print(tokenizer.decode(response, skip_special_tokens=True))
报错在outputs = model.generate()
原因分析：
刚开始怀疑是cuda题目，或者cpu、GPU题目
device = next(model.parameters()).device
print(f"Model is on {device}")
打印出来没有发现问题
解决方案：
在网上看到了一个解决方案，完美解决
torch.backends.cuda.enable_mem_efficient_sdp(False)
torch.backends.cuda.enable_flash_sdp(False)
原链接如下：https://stackoverflow.com/questions/77803696/runtimeerror-cutlassf-no-kernel-found-to-launch-when-running-huggingface-tran
免责声明：如果侵犯了您的权益，请联系站长，我们会及时删除侵权内容，谢谢合作！更多信息从访问主页：qidao123.com:ToB企服之家，中国第一个企服评测及商务社交产业平台。
页:
[1]