hadoop - Where to trace HDFS + Spark I/O calls? -


i'm trying trace i/o calls made spark hdfs research project. goal log of reads / writes / directory list commands additional metadata.

the expected output similar loggedfs posix file systems.

i tried put logging in org.apache.hadoop.fs of hadoop-hdfs-client project, doesn't seem spark using it.

what place put such logging spark requests traced?

thanks.


Comments

Popular posts from this blog

'hasOwnProperty' in javascript -

python - ValueError: No axis named 1 for object type <class 'pandas.core.series.Series'> -

java - How to provide dependency injections in Eclipse RCP 3.x? -