word-embedding models API cleaned up

Author: stephenhky

docs/codes.rst

@@ -97,12 +97,6 @@ Module `shorttext.utils.gensim_corpora`
 .. automodule:: shorttext.utils.gensim_corpora
    :members:
 
-Module `shorttext.utils.wordembed`
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-.. automodule:: shorttext.utils.wordembed
-   :members:
-
 Module `shorttext.utils.compactmodel_io`
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 

docs/tutorial_wordembed.rst

@@ -10,13 +10,12 @@ their page. To load the model, call:
 >>> import shorttext
 >>> wvmodel = shorttext.utils.load_word2vec_model('/path/to/GoogleNews-vectors-negative300.bin.gz')
 
-It is a binary file, and the default is set to be `binary=True`. In fact, it is equivalent to calling,
-if you have `gensim` version before 1.0.0:
+It is a binary file, and the default is set to be `binary=True`.
 
->>> import gensim
->>> wvmodel = gensim.models.Word2Vec.load_word2vec_format('/path/to/GoogleNews-vectors-negative300.bin.gz', binary=True)
+.. automodule:: shorttext.utils.wordembed
+   :members: load_word2vec_model
 
-Or beyond version 1.0.0,
+It is equivalent to calling,
 
 >>> import gensim
 >>> wvmodel = gensim.models.KeyedVectors.load_word2vec_format('/path/to/GoogleNews-vectors-negative300.bin.gz', binary=True)
@@ -87,6 +86,9 @@ To load a pre-trained FastText model, run:
 
 And it is used exactly the same way as Word2Vec.
 
+.. automodule:: shorttext.utils.wordembed
+   :members: load_fasttext_model
+
 Poincaré Embeddings
 -------------------
 
@@ -98,6 +100,8 @@ pre-trained model, run:
 
 For preloaded word-embedding models, please refer to :doc:`tutorial_wordembed`.
 
+.. automodule:: shorttext.utils.wordembed
+   :members: load_poincare_model
 
 BERT
 ----
@@ -120,6 +124,20 @@ The default BERT models and tokenizers are `bert-base_uncase`.
 If you want to use others, refer to `HuggingFace's model list
 <https://huggingface.co/models>`_ .
 
+.. autoclass:: shorttext.utils.transformers.BERTObject
+   :members:
+
+.. autoclass:: shorttext.utils.transformers.WrappedBERTEncoder
+   :members:
+
+
+Other Functions
+---------------
+
+.. automodule:: shorttext.utils.wordembed
+   :members: shorttext_to_avgvec
+
+
 Links
 -----
 

docs/tutorial_wordembedAPI.rst

@@ -32,6 +32,8 @@ using `RESTfulKeyedVectors`:
 
 This model can be used like other `gensim` `KeyedVectors`.
 
+.. autoclass:: shorttext.utils.wordembed.RESTfulKeyedVectors
+   :members:
 
 
 Home: :doc:`index`

shorttext/utils/wordembed.py

@@ -6,7 +6,7 @@
 from gensim.models.poincare import PoincareModel, PoincareKeyedVectors
 import requests
 
-from shorttext.utils import tokenize, deprecated
+from shorttext.utils import tokenize
 
 
 def load_word2vec_model(path, binary=True):