Handmade Dataset Experiments

Handmade Dataset Experiment #1: Generating Images

8% of your grade

Part I: Gathering Training Data:

You are welcome to use these prompts or do your own thing. Just make sure you have at least 300 images for next week. Don’t stress yourself out trying to make something perfect. Remember that this is just an experiment to familiarize yourself with the process.

Option 1:

Photograph a collection of similar objects (rocks, seashells, legos, jewelry, etc) against a solid color background.

  1. Pick a category of similar objects and gather as many individual examples as you can.
  2. Set up a solid-color background (i.e. use a big sheet of paper, or a blank wall) with even consistent lighting.
  3. Photograph each object separately. You’ll need about 300 photos but you likely don’t have 300 individual objects. So you can also think about photographing the same object from several different angles, turning it etc.
  4. The images will eventually be cropped square so try to be consistent about centering the object and using the same level of ‘zoom’ in each image.
  5. If you took these on your phone, transfer all 300 photos to your laptop before class next week.
  6. Submit your images in brightspace (i.e. as a zip folder).

Option 2:

Find a continuous movement (clouds drifting, a patch of sunlight across your floorboards, or an ice cube melting, etc.) and set up your phone/camera in such a way to film it. You will then split this video into ~300 frames to use as training images.

  1. Prop your camera up securely (using a small tripod, a phone stand, or a steady stack of books) and record the movement for about half an hour.
  2. Ideally, your subject matter is pretty homogenous. You don’t want a bunch of random stuff happening in the background. For example, if you are filming a leaf shaking in the wind, it would be better if the leaf was against a plain wall rather than with a whole park scene in the background.
  3. Convert your video to images using ->How to convert video to images in Adobe Media Encoder
  4. Have this video file and images on your laptop before class next week.
  5. Submit your images in brightspace (i.e. as a zip folder).

Part II

  1. Using the demo from class, finish training your GAN model. You may opt for the pix2pix version instead of the lightweight GAN. The pix2pix can be more forgiving with data quality but requires you to have an "A" dataset. Up to you. But be ready to share results (even if they are 'failed' results) with the class next week.
  2. Submit in brightspace your model file (.pict for pix2pix or .onnx for lightweight gan). If you have any issues, let me know over email please.

Handmade Dataset Experiment #2: Generating Text

8% of your grade

Gathering Data

You are welcome to use these prompts below or do your own thing. But you must not use copyrighted content. Make sure you have at least 10,000 words. Don’t stress yourself out trying to make something perfect. Remember that this is just an experiment to familiarize yourself with the process.

Prompt 1:

  1. Use text you’ve already written for school or personal creative writing.
  2. Your combined text file must be about 10,000 words long. (For example, if one essay isn't long enough, copy and paste two or three of your past papers into the same file).

Prompt 2:

  1. Open your email spam folder. Copy the text of many spam emails/texts and paste them into a single plain text file (.txt).
  2. Your combined text file must be about 10,000 words long.

Training Your Model

  1. Using the GPT-2 fine-tuning demo from class, finish training your text model on the dataset you gathered. Be ready to share results with the class next week. In Brightspace, submit a link to your trained model (the checkpoint folder).

Handmade Dataset Experiment #3: Generating Sound

8% of your grade

Part I: Gathering Data

Option 1:

  1. Record a routine noise with your phone or a recording device. For example, as you do work for another class on your computer, you could record the sound of your own typing. You could record yourself chopping vegetables.
  2. Try to get at least 2 hours of audio total.
  3. Try to reduce the amount of background/extraneous noise
  4. Move the audio from the phone/device to your laptop before next class.

Option 2:

  1. Record your own voice. You could be reading a text aloud, talking to someone, singing, or humming.
  2. Try to get at least 2 hours of audio total.
  3. Try to reduce the amount of background/extraneous noise
  4. Move the audio from the phone/device to your laptop before next class.

Option 3:

  1. If you play an instrument, record yourself playing it in a relatively quiet room.
  2. Try to get at least 2 hours of audio total.
  3. Try to reduce the amount of background/extraneous noise
  4. Move the audio from the phone/device to your laptop before next class.

Part II

  1. Using the demo from class, finish training your audio model. Be ready to share results with the class next week.