scala - Strange error I can't do anything with the data frame but it's not empty -
i'm running query in scala using databricks spark 1.6.0 (hadoop 1) filter url's data have in redshift once query finish successfully, if run count on data frame shows there data on data frame try display data or join data data frame appears empty, doesn't display , can't make join.
this code getting data databricks
val df = sqlcontext.read .format("com.databricks.spark.redshift") .option("url", jdbcurl) .option("tempdir", s"s3a://....") .option("query", s"select * table column '%word1%word2%word3%' ) .load() the actual data this
'https://www.asdfg.com/word1?word2=/word3/asdasdadasd' or
'https://www.asdfg.com/word1?word2=%2fword3%2fasdasdadasd' i can't understand why if run count results other operation data frame appears empty, ideas why happening?
Comments
Post a Comment