tensorflow - What is the default kernel initializer in tf.layers.conv2d and tf.layers.dense? -

the official tensorflow api doc claims parameter kernel_initializer defaults none tf.layers.conv2d , tf.layers.dense.

however, reading layers tutorial (https://www.tensorflow.org/tutorials/layers), noted parameter not set in code. example:

# convolutional layer #1   conv1 = tf.layers.conv2d(       inputs=input_layer,       filters=32,       kernel_size=[5, 5],       padding="same",       activation=tf.nn.relu) 

the example code tutorial runs without errors, think default kernel_initializer not "none". so, initializer used?

in code, did not set kernel_initializer of conv2d , dense layers, , fine. however, when tried set kernel_initializer tf.truncated_normal_initializer(stddev=0.1, dtype=tf.float32), got nan errors. going on here? can help?

great question! quite trick find out!

• as can see, not documented in tf.layers.conv2d
• if @ definition of the function see function calls variable_scope.get_variable:

in code:

self.kernel = vs.get_variable('kernel',                                   shape=kernel_shape,                                   initializer=self.kernel_initializer,                                   regularizer=self.kernel_regularizer,                                   trainable=true,                                   dtype=self.dtype) 

next step: what variable scope when initializer none?

here says:

if initializer none (the default), default initializer passed in constructor used. if 1 none too, use new glorot_uniform_initializer.

so answer is: uses glorot_uniform_initializer

for completeness definition of initializer:

the glorot uniform initializer, called xavier uniform initializer. draws samples uniform distribution within [-limit, limit] limit sqrt(6 / (fan_in + fan_out)) fan_in number of input units in weight tensor , fan_out number of output units in weight tensor. reference: http://jmlr.org/proceedings/papers/v9/glorot10a/glorot10a.pdf

edit: found in code , documentation. perhaps verify initialization looks running eval on weights!