Skip to content

compression

Jump to bottom
Huangkai ZHENG edited this page Nov 6, 2019 · 27 revisions

Code compression

ALiX is a javascript project developped on eclipse.Besides it's designed as an widget which can be implanted in any other existing protal independant of any specific data source. Therefore it need to be well packaged so as to facilitate the import.The code compression is divided in two parts.

- Compression of javascript

We use LMpack to compress the js files

1. Why

We try to use webpack to compress js files, but it's very difficult to rebuild the configuration file each time we do some modification for the project. LMpack is a code compress tool designed by Laurent Michel. It is based on Linux shell and it is very efficient.

2. How
  • Create minifier.bash

First of all, we should create a file minifier.bash in our project. The file structure in Alix is as below.

  • lmpack
    • alix_packed
      • packedJS_alix.js
      • packedJSmin_alix.js
      • packedJSimport.js
      • packedJSimportmin.js
    • minifier
      • minifier.bash
  • Declare the file path It's important to declare the path of file input and output.
alixDir="${script_dir}/../../.."
outputDir="${script_dir}/../alix_packed" # directory where both packed JS and CSS are stored 
packedJS=$outputDir/packedJS_alix.js       # name of the file containing the packed JS
packedJSmin=$outputDir/packedJSmin_alix.js 
packedJSimport=$outputDir/packedJSimport_alix.js       # name of the file containing the packed JS
packedJSimportmin=$outputDir/packedJSimportmin_alix.js
  • Add the js files in minifier.bash, there are 2 array which are used to save the js file, js_array_org and jsimport_array_org.
js_array_org=(
		 "aladin.js"
		 "AladinUpdate.js" 
		 "Modalinfo.js"
		 "Out.js"
		 "packedLoader.js"
		 "PageLocation.js"
		 "Printer.js"
		 "Processing.js"
		 "SkyGeometry.js"
		 "Segment.js"
	   	 "AstroCoo.js"
	   	 "LibraryMap.js"
	   	 "LibraryCatalog.js"
	   	 "MasterResource.js"
	   	 "AladinLiteView.js"
	   	 "AladinLite_v.js"
	   	 "AladinLite_c.js"
	   	 "Historique_m.js"
	   	 "Historique_v.js"	
	   	 "RegionEditor_v.js"
	   	 "RegionEditor_m.js"
	   	 "RegionEditor_c.js"
	   	 "HipsSelector_m.js"
	   	 "HipsSelector_v.js" 
	   	 "SwarmDynamicFilter.js"
	   	 "ConfigureALiX.js"
	   	 "VizierCatalog.js"
	   	 "MessageBox.js"
	   	 "SimbadCatalog.js"
	   	 "CustomDataTable.js"
	   	 "ModalResult.js"
          )   
          
jsimport_array_org=( 
	   	 "spectrum.js"
	   	 "jquery-ui.js"
	     "jquery.ui.dialog.js"
	     "jquery.simplemodal.js"
	     "jquery.alerts.js"
	     "jquery.dataTables.js"
	     "FixedHeader.js"
	     "jquery.prints.js"
	     "jquery.tooltip.js"
	     "jquery.form.js"
	     "jquery-migrate-1.4.1.js"
          )
  • Compile the js files

In this step, we need to copy the js files to the output folder. There are 2 functions minifySet and minifySet2. One is for local js files and the other is for external files.

js_array=() # list of packed js files
function minifySet(){
	inputDir=$1
	shift
	fileList=("$@")
	for item in "${fileList[@]}"
	do
		echo compiling $inputDir/${item} to $outputDir/${item}
		#
		# Closure compilation is commented whole the validation phase
    	#java -jar compiler.jar --js $inputDir/${item} --js_output_file $outputDir/${item} || exit 1
    	cp $inputDir/${item}  $outputDir/${item} || exit 1
    	# Store an ordered list of minified files
    	js_array[${#js_array[@]}]=$item
	done
}

js_array2=() # list of packed js files
function minifySet2(){
	inputDir=$1
	shift
	fileList=("$@")
	for item in "${fileList[@]}"
	do
		echo compiling $inputDir/${item} to $outputDir/${item}
		#
		# Closure compilation is commented whole the validation phase
    	#java -jar compiler.jar --js $inputDir/${item} --js_output_file $outputDir/${item} || exit 1
    	cp $inputDir/${item}  $outputDir/${item} || exit 1
    	# Store an ordered list of minified files
    	js_array2[${#js_array2[@]}]=$item
	done
}
  • Merge all minified JS files

In this part, we need to merge all the JS files and then remove them from the output folder.

function  pack() {
	rm -f $outputFile
	for item in "${js_array[@]}"
	do 
		echo pack $outputDir/$item to $packedJS
		cat $outputDir/$item >> $packedJS || exit 1
		echo "" >> $packedJS
		echo "console.log('=============== > " $item "');" >> $packedJS
		echo "" >> $packedJS
		rm $outputDir/$item
	done
}	

function  packimport() {
	rm -f $outputFile
	for item in "${js_array2[@]}"
	do 
		echo packimport $outputDir/$item to $packedJSimport
		cat $outputDir/$item >> $packedJSimport || exit 1
		echo "" >> $packedJSimport
		echo "console.log('=============== > " $item "');" >> $packedJSimport
		echo "" >> $packedJSimport
		rm $outputDir/$item
	done
}
  • Script Job

We have defined all the variables and functions. So how to use it ? Now we need to write the shell statement. Firstly, we need to pack the JS files.

pwd
echo "=========== copy JS files"
rm -f $packedJS
minifySet "${alixDir}/javascript"   \
    ${js_basic_array[@]} 

echo "=========== Pack JS  files"
pack 

echo "=========== copy JSimport files"
rm -f $packedJSimport
minifySet2 "${alixDir}/jsimports"   \
    ${js_import_array[@]} 
    
echo "=========== Pack JS import files"
packimport 

echo "=========== Packing is over"

Then, considering that the packed files are too large, we use a jar compress tool to compress the code.

echo "=========== Compress JS"
java -jar ${script_dir}/compiler.jar --js=$packedJS --js_output_file=$packedJSmin
echo "=========== Compress is over"

echo "=========== Compress JS import"
java -jar ${script_dir}/compiler.jar --js=$packedJSimport --js_output_file=$packedJSimportmin
echo "=========== Compress is over"

- Compression of styles(css)

We use webpack to compress the css files.

1. Why

Webpack is a module builder that runs during our development rather than during the page.

It's a build tool that puts all of our assets, including Javascript, images, fonts, and CSS, in a dependency graph. Webpack use ==require()== in the source code (main.js) to point to local files, like images, and decide how they're processed in our final JS/CSS bundle, like replacing the path with a URL.

  • Dead asset elimination. This is killer, especially for CSS rules. You only build the images and CSS into your dist/ folder that your application actually needs.That's why we choose it.

  • Easier code splitting. For example, because you know that your file Homepage.js only requires specific CSS files, Webpack could easily build a homepage.css file to greatly reduce initial file size.

  • To control how assets are processed. If an image is below a certain size, you could base64 encode it directly into your Javascript for fewer HTTP requests.

  • Stable production deploys. You can't accidentally deploy code with images missing, or outdated styles.

Webpack will slow you down at the start，cause configuring Webpack is a minefield for newcomers,the configuration file syntax is confusing, but give you great speed benefits when used correctly as above-mentioned.

2. How
  • Install the webpack
//Global Installation
> npm install -g webpack
//Local Installation
> npm install webpack --save-dev

It's well explained in the url following and be prepared for a little long configuration in the begining if you need so many powerful modules of webpack.

https://webpack.js.org/guides/installation/

  • Run it
> npm start

Once we run the webpack, it goes into our entry point (main.js) searches and figures out what we required, in which order we needs it, and what each piece depends on such as the images for css. It will then create bundles of the files chosen as few as possible, as optimized as possible.

We can packed js and css seperately by install the ==extract-text-webpack-plugin==. Here in this project we've only packed the css files.

// use npm 
>npm install extract-text-webpack-plugin --save-dev
//webpack.config.js
const ExtractTextPlugin = require("extract-text-webpack-plugin");
module.exports = {
		
  mode: 'development',

 target: 'node',
	  node: {
		   fsevents: "empty",
		   fs: "empty",
		   child_process: "empty",
		   //global: false
		},
  entry: __dirname+"/app/Main.js",//The only access file .Only!!!
  output: {
   path:__dirname+"/public",//where to put the bundle file
    filename: "bundle.js",
   // library: "my-library",
   // libraryTarget: "umd"
  },
......,
  module: {
    rules: [
      {
        test: /\.css$/,
        use: ExtractTextPlugin.extract({
          fallback: "style-loader",
          use: "css-loader"
        })
      }
    ]
  },
  plugins: [
    new ExtractTextPlugin("style_bundle.css"),
  ]
}
1. Import the files we need in the entry js.
//packager/webpack/app/main.js
import '../../../styleimport/jquery-ui.min.css';	   
import '../../../styleimport/bootstrap-3.3.7/bootstrap.min.css';
import '../../../styleimport/menuDemo/livedemo.css';
import   "../../../styleimport/jquery.ui.theme.css";
import	 "../../../styleimport/jquery.ui.dialog.css";
import	 "../../../styleimport/spectrum.css";
2.Enter where the node_modules is located and then call the 'npm start'.

For example , here the root directory is /packager/webpack

The result 'style_bundle' will be put into the /packager/webpack/public as configured in the webpack.config.js

3. Result

In ALiX , we packed all import css files in the style_bundle.css. The css specific for ALiX 'aladinliteX.css' is not included so as to alter and optimize the style more easily. So if you want to integrate ALiX in your own project, don't forget to import the three css highlight files following.

  • aladin.css
  • aladinliteX.css
  • style_bundle.css

style_bundle includes these css following

  • jquery-ui.min.css
  • bootstrap.min.css
  • livedemo.css
  • jquery.ui.theme.css
  • jquery.ui.dialog.css
  • spectrum.css

aladinlitX.css needs to be the last one to load.

Clone this wiki locally