python - TfIdfVectorizer with grouped tokens -


i've vectorized ~1million of textual documents tfidfvectorizer.

vect1 = tfidfvectorizer() vect1.fit_transform(data)

now need group (semantically similar) tokens , repeat vectorization.

replacing tokens in raw data new ones seems tedious , time consuming me, since intend repeat process multiple grouping strategies.

i've tried 

voc = vect1.vocabulary_ voc['cabman'] = voc['driver'] # grouping similar tokens have same ids ... vect2 = tfidfvectorizer(vocabulary=voc) vect2.fit_transform(data)

valueerror: vocabulary contains repeated indices.

is there efficient way accomplish without touching input data?


Comments

Post a Comment

Popular posts from this blog

'hasOwnProperty' in javascript -

i use obj.hasownproperty judge whether object has property, when replaced obj[prop] !== undefined , not normal implementation, ask, why behind method can not use it? object.hasownproperty(prop); object[prop] !== undefined; obj[prop] !== undefined wrong 2 reasons: you can explicitly set property undefined , obj[prop] = undefined; . obj.hasownproperty(prop) return true in case. obj[prop] follow prototype chain, return property that's inherited. obj.hasownproperty(prop) returns true if property exists directly in object, returns false inherited properties.
Read more

python - ValueError: No axis named 1 for object type <class 'pandas.core.series.Series'> -

i´m new programming. i´m trying use scipy minimize, had several issues , gotten through of them. right code, i'm not understanding why i´m getting error. par_opt = so.minimize(fun=fun_obj, x0=par_ini, method='nelder-mead', args=[series_pt_cal, dt, series_caudal_cal]) not enough info given op, somewhere in code it's specified operate data frame column (axis=1) on object pandas series. if code typically works occasional gives errors, check degenerative cases data frame may have 1 row. pandas has nasty habit of guessing want -- may decide reduce 1-row data frame series (e.g., apply() function; can disable using reduce=false in there). add line of code check object isinstance(df, pd.dataframe) or else convert offending pandas series data frame, s.to_frame().t problems had deal with.
Read more

Command prompt result in label. Python 2.7 -

i want put result(output) of shell command label. command works, label shows "0", however, in command prompt, result shown correctly, need showed in label. doing ubuntu. myg1 = button(root, text="rodyti informacija", command=lambda: gauti()) myg1.pack(side=bottom) def gauti(): imti = tekstas.get("1.0", "end-1c") info = subprocess.call("id '{imti}' ".format(imti=imti), shell=true) w = label(root, text= info) w.pack(side = bottom) subprocess.call() returns exit code of process created (in case 0 success). if want text output of process, should instead call subprocess.check_output() .
Read more