Fine-tune and collect data for fine-tuning OPT models

Created by: Behnamhb

hey dear meta developers

problem: I want to fine-tune OP3 30B model with new datasets.what is the template for generate dataset ? example : [key] = [value].Im reading your code and find a script for this goal.streaming_finetune_language_modeling.py I see a jsonl format and I can't understand what is it and how can I generate dataset .

my second problem: I have 2 gpus (v100 32G) on a dedicated server .can I fine-tune 30B model with my hardware or not?

my third problem: can we use 30B for transfer learning ?

again thank you for answer my questions.