{"id":1598,"date":"2021-09-13T13:34:33","date_gmt":"2021-09-13T04:34:33","guid":{"rendered":"https:\/\/www.kwonline.org\/memo2\/?p=1598"},"modified":"2021-09-13T13:35:59","modified_gmt":"2021-09-13T04:35:59","slug":"azure-databricks-standard-tier-to-access-adls2","status":"publish","type":"post","link":"https:\/\/www.kwonline.org\/memo2\/2021\/09\/13\/azure-databricks-standard-tier-to-access-adls2\/","title":{"rendered":"Azure Databricks (Standard Tier) \u3067 ADLS2 \u306b\u30a2\u30af\u30bb\u30b9\u3059\u308b"},"content":{"rendered":"

 
\ndoc \u306b\u306f KeyVault + SecretScope \u4f7f\u3048\u3063\u3066\u66f8\u3044\u3066\u3042\u308b\u3051\u3069 Standard Tier \u3060\u3068 SecretScope \u4f7f\u3048\u3093\u304b\u3063\u305f\u306e\u3067\u30e1\u30e2\u3002<\/p>\n

Azure Data Lake Storage Gen2-Azure Databricks \u3092\u4f7f\u3063\u3066\u307f\u308b – Workspace | Microsoft Docs<\/a><\/p>\n

SharedKey\u4f7f\u3063\u3066\u3082\u8aad\u307f\u8fbc\u3081\u308b\u3002<\/p>\n

\r\nval acc = "\u30b9\u30c8\u30ec\u30fc\u30b8\u30a2\u30ab\u30a6\u30f3\u30c8"\r\nval cont = "\u30b3\u30f3\u30c6\u30ca"\r\nval adls_path = "abfss:\/\/" + cont + "@" + acc + ".dfs.core.windows.net\/"\r\n\r\nspark.conf.set("fs.azure.account.auth.type."+ acc +".dfs.core.windows.net", "SharedKey")\r\nspark.conf.set("fs.azure.account.key."+ acc +".dfs.core.windows.net", "\u30ad\u30fc")\r\n\r\nval df = spark.read.option("header", "true").option("delimiter", ",").csv(adls_path + "myfile.csv")\r\n\r\ndisplay(df)\r\n<\/pre>\n
\u3042\u3068\u306f blob \u3068\u3057\u3066\u30de\u30a6\u30f3\u30c8\u3059\u308b\u65b9\u6cd5\u3082\u53ef\u3002<\/p>\n
\r\nval acc = "\u30b9\u30c8\u30ec\u30fc\u30b8\u30a2\u30ab\u30a6\u30f3\u30c8"\r\nval cont = "\u30b3\u30f3\u30c6\u30ca"\r\nval blob_path = "wasbs:\/\/" + cont + "@" + acc + ".blob.core.windows.net\/"\r\n\r\ndbutils.fs.mount(\r\n  source = blob_path\r\n  mountPoint = "\/mnt\/data",\r\n  extraConfigs = Map("fs.azure.account.key." + acc + ".blob.core.windows.net" -> "\u30ad\u30fc"))\r\n\r\nval df = spark.read.option("header", "true").option("delimiter", ",").csv("\/mnt\/data\/myfile.csv")\r\n\r\ndisplay(df)\r\n<\/pre>\n
 <\/p>\n","protected":false},"excerpt":{"rendered":"
  doc \u306b\u306f KeyVault + SecretScope \u4f7f\u3048\u3063\u3066\u66f8\u3044\u3066\u3042\u308b\u3051\u3069 Standard Tier \u3060\u3068 SecretScope \u4f7f\u3048\u3093\u304b\u3063\u305f\u306e\u3067\u30e1\u30e2\u3002 Azure Data Lake Stor […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[21,9,10],"tags":[],"class_list":["post-1598","post","type-post","status-publish","format-standard","hentry","category-data-engineering","category-scala","category-spark"],"_links":{"self":[{"href":"https:\/\/www.kwonline.org\/memo2\/wp-json\/wp\/v2\/posts\/1598","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kwonline.org\/memo2\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kwonline.org\/memo2\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kwonline.org\/memo2\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kwonline.org\/memo2\/wp-json\/wp\/v2\/comments?post=1598"}],"version-history":[{"count":5,"href":"https:\/\/www.kwonline.org\/memo2\/wp-json\/wp\/v2\/posts\/1598\/revisions"}],"predecessor-version":[{"id":1603,"href":"https:\/\/www.kwonline.org\/memo2\/wp-json\/wp\/v2\/posts\/1598\/revisions\/1603"}],"wp:attachment":[{"href":"https:\/\/www.kwonline.org\/memo2\/wp-json\/wp\/v2\/media?parent=1598"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kwonline.org\/memo2\/wp-json\/wp\/v2\/categories?post=1598"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kwonline.org\/memo2\/wp-json\/wp\/v2\/tags?post=1598"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}