[Fun] compress and composite dataset to one image file

If you want to compress and composite the images in a dataset, like MNIST or CIFAR-10, you probably can refer to my example.

People need to see it to believe it. Here you go:



<CIFAR-10>

   <MNIST>

My approach is based on this tool: imagemagick. You can check my previous post for installation and usage: https://danny270degree.blogspot.com/2016/09/image-how-to-resize-convert-modify.html

So, in the beginning, you should transfer your dataset to raw image files. For this, you can find out the way to do it because this post doesn't focus on transferring.

Second, go into your directory which contains all your folders with images or contains image files directly. Then, you run the following python code and it will generate a destination name "dest.png" for the result.

import os

COLUMN_LENGTH = 100 #the number of images for column in destination file
EXT_FILE = ".png"   #the sub extendion image file name that we need to process
dest_path = "/tmp"  #where to put your destination and temp files
dest_name = "dest"  #your destination file name

"""
Compress your target images files to dest_path
"""
def CompressImage(root, file, dest_path):
  os.system('convert %s -resize 14x14 %s' % (os.path.join(root, file), 
                                                 os.path.join(dest_path, file)))
"""
Composite your compressed images files to destination file
"""
def CompositeImage(rawfile_list, dest_path, dest_name):
  global COLUMN_LENGTH
  global EXT_FILE

  dest_vertical_list = []
  command_header = "convert +append "
  command_body = ""
  #vertical composite
  for i in range(len(rawfile_list)//COLUMN_LENGTH):
    for j in range(COLUMN_LENGTH):
      command_body += rawfile_list[i*COLUMN_LENGTH + j] + " "
    dest_file = os.path.join(dest_path, "c" + str(i) + EXT_FILE)
    command_body = command_header + command_body + dest_file
    dest_vertical_list.append(dest_file)
    os.system(command_body)
    command_body = ""
  
  #horizontal composite
  command_header = "convert -append "
  command_body = ""
  for i in dest_vertical_list:
    command_body += i + " "
  command_body = command_header + command_body + os.path.join(dest_path, dest_name + EXT_FILE)
  os.system(command_body)


# do compression
for root, dirs, files in os.walk("./"):
  for file in files:
    if file.endswith(EXT_FILE):
      print(os.path.join(root, file))
      CompressImage(root, file, dest_path)

# do composite
rawfile_list = []
for root, dirs, files in os.walk("./"):
 for file in files:
  if file.endswith(EXT_FILE):
    #print(os.path.join(root, file))
    rawfile_list.append(os.path.join(root, file))
CompositeImage(rawfile_list, dest_path, dest_name)

P.S: you also should modify the global variables, like EXT_FILES, dest_path, or dest_name, which depends on your images' ext name and your working space's path.