New Post has been published on https://semicolon.codes/sketchcode-generating-html-code-hand-drawn-wireframe/
SketchCode - Image into HTML Codes
SketchCode is a deep learning model that takes hand-drawn web mockups and converts them into working HTML code which uses an image captioning architecture to generate its HTML markup from hand-drawn website wireframes.
This project builds on the synthetically generated dataset and model architecture from pix2code by Tony Beltramelli and the Design Mockups project from Emil Wallner.
Note: This project is meant as a proof-of-concept so, the model isn’t (yet) built to generalize to the variability of sketches seen in actual wireframes and thus its performance relies on wireframes resembling the core dataset.
Python 3 (not compatible with python 2)
pip install -r requirements.txt
Download the data and pretrained weights:
# Getting the data, 1,700 images, 342mb git clone https://github.com/ashnkumar/sketch-code.git cd sketch-code cd scripts # Get the data and pretrained weights sh get_data.sh sh get_pretrained_model.sh
Converting an example drawn image into HTML code, using pretrained weights:
cd src python convert_single_image.py --png_path ../examples/drawn_example1.png \ --output_folder ./generated_html \ --model_json_file ../bin/model_json.json \ --model_weights_file ../bin/weights.h5
Converting a single image into HTML code, using weights:
cd src python convert_single_image.py --png_path path/to/img.png \ --output_folder folder/to/output/html \ --model_json_file path/to/model/json_file.json \ --model_weights_file path/to/model/weights.h5
Converting a batch of images in a folder to HTML:
cd src python convert_batch_of_images.py --pngs_path path/to/folder/with/pngs \ --output_folder folder/to/output/html \ --model_json_file path/to/model/json_file.json \ --model_weights_file path/to/model/weights.h5
cd src # training from scratch # <augment_training_data> adds Keras ImageDataGenerator augmentation for training images python train.py --data_input_path path/to/folder/with/pngs/guis \ --validation_split 0.2 \ --epochs 10 \ --model_output_path path/to/output/model --augment_training_data 1 # training starting with pretrained model python train.py --data_input_path path/to/folder/with/pngs/guis \ --validation_split 0.2 \ --epochs 10 \ --model_output_path path/to/output/model \ --model_json_file ../bin/model_json.json \ --model_weights_file ../bin/pretrained_weights.h5 \ --augment_training_data 1
Evalute the generated prediction using the BLEU score
cd src # evaluate single GUI prediction python evaluate_single_gui.py --original_gui_filepath path/to/original/gui/file \ --predicted_gui_filepath path/to/predicted/gui/file # training starting with pretrained model python evaluate_batch_guis.py --original_guis_filepath path/to/folder/with/original/guis \ --predicted_guis_filepath path/to/folder/with/predicted/guis
Don’t worry, File will be unlocked as soon as the following task is completed.
jQuery(function()jQuery('.link-btn a.wpdm-download-link img').after('<img src="https://semicolon.codes/wp-content/plugins/wpdm-download-button/images/03.png" alt="" />');jQuery('.link-btn a.wpdm-download-link img').remove(););