finished documention on various neural networks; added references

Author: stephenhky

docs/images/nnlib_clstm.png

@@ -39,4 +39,8 @@ WWW '08 Proceedings of the 17th international conference on World Wide Web. (200
 <http://dl.acm.org/citation.cfm?id=1367510>`_]
 
 Yoon Kim, "Convolutional Neural Networks for Sentence Classification," *EMNLP* 2014, 1746-1751 (arXiv:1408.5882). [`arXiv
-<https://arxiv.org/abs/1408.5882>`_]
+<https://arxiv.org/abs/1408.5882>`_]
+
+Zackary C. Lipton, John Berkowitz, "A Critical Review of Recurrent Neural Networks for Sequence Learning," arXiv:1506.00019 (2015). [`arXiv
+<https://arxiv.org/abs/1506.00019>`_]
+

docs/images/nnlib_cnn.png

@@ -37,7 +37,7 @@ Then load the training data
 Then we choose a neural network. We choose ConvNet:
 
 >>> import shorttext.classifiers.embed.nnlib.frameworks as fr
->>> kmodel = fr.CNNWordEmbed(len(classdict.keys()))
+>>> kmodel = fr.CNNWordEmbed(len(trainclassdict.keys()))
 
 Initialize the classifier:
 
@@ -69,18 +69,88 @@ Epoch 10/10
 45/45 [==============================] - 0s - loss: 0.0743
 
 Then the model is ready for classification, like:
+
 >>> classifier.score('artificial intelligence')
 {'mathematics': 0.57749695, 'physics': 0.33749574, 'theology': 0.085007325}
 
 Provided Neural Networks
 ------------------------
 
+There are three neural networks available in this package for the use in
+:class:`shorttext.classifiers.embed.nnlib.VarNNEmbeddedVecClassification.VarNNEmbeddedVecClassifier`,
+and they are available in the module :module:`shorttext.classifiers.embed.nnlib.frameworks`.
+
+ConvNet (Convolutional Neural Network)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This neural network for supervised learning is using convolutional neural network (ConvNet),
+as demonstrated in Kim's paper.
+
+.. image:: images/nnlib_cnn.png
+
+The function in the frameworks returns a :class:`keras.models.Sequential`.
+
+.. autofunction:: shortext.embed.nnlib.frameworks.CNNWordEmbed
+
+The parameter `maxlen` defines the maximum length of the sentences. If the sentence has less than `maxlen`
+words, then the empty words will be filled with zero vectors.
+
+>>> kmodel = fr.CNNWordEmbed(len(trainclassdict.keys()))
+
+Double ConvNet
+^^^^^^^^^^^^^^
+
+This neural network is nothing more than two ConvNet layers.
+
+.. autofunction:: shortext.embed.nnlib.frameworks.DoubleCNNWordEmbed
+
+The parameter `maxlen` defines the maximum length of the sentences. If the sentence has less than `maxlen`
+words, then the empty words will be filled with zero vectors.
+
+>>> kmodel = fr.DoubleCNNWordEmbed(len(trainclassdict.keys()))
+
+C-LSTM (Convolutional Long Short-Term Memory)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This neural network for supervised learning is using C-LSTM, according to the paper
+written by Zhou *et. al.* It is a neural network with ConvNet as the first layer,
+and then followed by LSTM (long short-term memory), a type of recurrent neural network (RNN).
+
+.. image:: images/nnlib_clstm.png
+
+The function in the frameworks returns a :class:`keras.models.Sequential`.
+
+.. autofunction:: shorttext.embed.nnlib.frameworks.CLSTMWordEmbed
+
+The parameter `maxlen` defines the maximum length of the sentences. If the sentence has less than `maxlen`
+words, then the empty words will be filled with zero vectors.
+
+>>> kmodel = fr.CLSTMWordEmbed(len(trainclassdict.keys()))
+
+User-Defined Neural Network
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Users can define their own neural network for use in the classifier wrapped by
+:class:`shorttext.classifiers.embed.nnlib.VarNNEmbeddedVecClassification.VarNNEmbeddedVecClassifier`
+as long as the following criteria are met:
+
+- the input matrix is :class:`numpy.ndarray`, and of shape `(maxlen, vecsize)`, where
+`maxlen` is the maximum length of the sentence, and `vecsize` is the number of dimensions
+of the embedded vectors. The output is a one-dimensional array, of size equal to
+the number of classes provided by the training data. The order of the class labels is assumed
+to be the same as the order of the given training data (stored as a Python dictionary).
 
 Reference
 ---------
 
 Chunting Zhou, Chonglin Sun, Zhiyuan Liu, Francis Lau, "A C-LSTM Neural Network for Text Classification," (arXiv:1511.08630). [`arXiv
 <https://arxiv.org/abs/1511.08630>`_]
 
+"CS231n Convolutional Neural Networks for Visual Recognition," Stanford Online Course. [`link
+<http://cs231n.github.io/convolutional-networks/>`_]
+
 Yoon Kim, "Convolutional Neural Networks for Sentence Classification," *EMNLP* 2014, 1746-1751 (arXiv:1408.5882). [`arXiv
 <https://arxiv.org/abs/1408.5882>`_]
+
+Zackary C. Lipton, John Berkowitz, "A Critical Review of Recurrent Neural Networks for Sequence Learning," arXiv:1506.00019 (2015). [`arXiv
+<https://arxiv.org/abs/1506.00019>`_]

docs/refs.rst

@@ -0,0 +1 @@
+import SumWord2VecClassification