1import{B as c,Dt as e,Ot as o,R as t,U as p,a as K,c as q,ct as M,lt as n,st as r}from"../chunks/DyEG9IH5.js";import"../chunks/DwdwGuwu.js";import"../chunks/XPhOh-Be.js";import{t as i}from"../chunks/CKq3EId7.js";import{t as Y}from"../chunks/B-CG_ua0.js";var P={title:"Audit Your S3 Storage Immediately",description:"A Quick and Dirty Guide",author:"Patrick Walsh",date:"2017-07-21T16:38:17.190Z",image:"/images/blog/amazon-free-data-hero.jpg",socialImage:"/images/blog/amazon-free-data-social.jpg",categories:["data-privacy","security"],keywords:[],canonical:"https://blog.ironcorelabs.com/audit-your-s3-storage-immediately-2b2e10e63b8f"},{title:ra,description:ia,author:la,date:pa,image:ca,socialImage:ua,categories:da,keywords:ha,canonical:ka}=P,H=c('<span class="icon icon-link"></span>'),J=c('<span class="icon icon-link"></span>'),Q=c('<span class="icon icon-link"></span>'),Z=c('<span class="icon icon-link"></span>'),X=c(`<p><strong><em>UPDATES:</em></strong> <em>more S3 leaks added to the bottom of the article.</em></p> <p>In the past month, weâve seen breach after breach hit huge organizations not because some clever hacker used a zero-day exploit, but because data was stored in Amazonâs S3 data storage <strong>and the permissions allowed anyone to read the data</strong>.</p> <p>Some of these organizations worked with consultants or third-party vendors who misconfigured the data protections, but that doesnât absolve them of responsibility. In fact, you canât delegate responsibility (just authority), and so a breach of an organizationâs data, even via a partner, is still ultimately the fault of the organization. Post breach lawsuits prove this time and again.</p> <h4 id="security-by-obscurity"><!>Security by Obscurity</h4> <p>The URLs that point to S3 buckets have a lot of crazy characters in them that frequently arenât linked publicly. So itâs easy for someone to post something publicly, but expect that no one will ever find it. Unfortunately, this is almost never true. Obscurity can work in the short term, but pretty much never works in the long term. Hoping that no one discovers the magic URL that leads to all of your customer data is a very bad idea.</p> <p>In the case of S3, there are numerous public tools, such as DigiNinjaâs Bucket Finder, that scan Amazon for public S3 buckets with hidden data.</p> <p>Here are some organizations who learned this lesson the hard way in the past couple of months:</p> <ul><li><strong>Republican National Committee</strong>âA vendor put the personal information and GOP analysis of more than <strong>198 million voters</strong> on AWS S3 in a bucket shared with the public.</li> <li><strong>Booz Allen Hamiltonâ</strong>ââThe folks responsible for some of the biggest Government leaks in recent history, including the Snowden leak, got hit again when a trove of <strong>top secret</strong> data was found unprotected on S3. Over <strong>60,000 files</strong> were publicly accessible.</li> <li><strong>World Wrestling Entertainment (WWE)</strong>âââPersonal data on <strong>3 million fans</strong> was stored unsecured on S3. Information included home addresses, educational background, earnings, and ethnicity.</li> <li><strong>Verizon</strong>âThe personal data of <strong>6 million customersâ</strong> was posted by a partner on S3 and made public. The data included customer names, mobile numbers, account PINs, home addresses, emails, and account balances.</li> <li><strong>Dow Jonesâ</strong>ââData on between <strong>2.2 million and 4 million customers</strong> (the correct number is a subject of dispute) was exposed to any logged-in AWS user. There are millions of such users. Data included names, addresses, and partial credit card numbers. Also exposed were databases of sensitive compliance information from partner financial institutions.</li></ul> <h4 id="amazon-warns"><!>Amazon Warns</h4> <p>Amazon sent out a notice to S3 users with buckets of files that are publicly readable. That warning reads in part:</p> <blockquote><p>Weâre writing to remind you that one or more of your Amazon S3 bucket access control lists (ACLs) are currently configured to allow access from any user on the internet.</p></blockquote> <p>If you maintain an Amazon S3 bucket, whether you received the notice or not, it behooves you to check the permissions on your buckets and the objects in those buckets as well. Instructions are below.</p> <h4 id="auditing-yourbuckets"><!>Auditing Your Buckets</h4> <p>It isnât too hard to double check your bucket permissions using <!>. Check out their site to figure out how best to install it, but on a Mac, <code>brew install aws</code> works nicely.</p> <p>
1Next, configure the tool with your AWS S3 credentials by calling <code>aws configure</code>.</p> <p>Finally, iterate over each bucket and print out the permissions using this little bash script you can copy into a terminal:</p> <pre class="language-bash"><div class="language-label language-label-bash">bash</div><code class="language-bash"> 2<span class="token assign-left variable"><span class="token environment constant">IFS</span></span><span class="token operator">=</span><span class="token variable"><span class="token variable">$(</span><span class="token builtin class-name">echo</span> <span class="token parameter variable">-en</span> <span class="token string">"<span class="token entity" title="\\\\">\\\\</span>n<span class="token entity" title="\\\\">\\\\</span>b"</span><span class="token variable">)</span></span> 3<span class="token keyword">for</span> <span class="token for-or-select variable">bucket</span> <span class="token keyword">in</span> <span class="token punctuation">\\</span><span class="token variable"><span class="token variable">\`</span>**aws s3 ls** <span class="token operator">|</span> <span class="token function">awk</span> <span class="token string">'{ print $3 }'</span><span class="token punctuation">\\</span><span class="token variable">\`</span></span> 4 <span class="token keyword">do</span> <span class="token builtin class-name">echo</span> <span class="token string">"Bucket <span class="token variable">$bucket</span>:"</span> 5 **aws s3api get-bucket-acl** <span class="token parameter variable">--bucket</span> <span class="token string">"<span class="token variable">$bucket</span>"</span> 6<span class="token keyword">done</span> 7</code></pre> <p>You should see output something like this:</p> <pre class="language-json5"><div class="language-label language-label-json5">JSON</div><code class="language-json5"> 8Bucket blah-blah-<span class="token property unquoted">blah</span><span class="token operator">:</span> 9<span class="token punctuation">{</span> 10 <span class="token property">"Owner"</span><span class="token operator">:</span> <span class="token punctuation">{</span> 11 <span class="token property">"DisplayName"</span><span class="token operator">:</span> <span class="token string">"someuser"</span><span class="token punctuation">,</span> 12 <span class="token property">"ID"</span><span class="token operator">:</span> <span class="token string">"8942fa787d23f..."</span> 13 <span class="token punctuation">}</span><span class="token punctuation">,</span> 14 <span class="token property">"Grants"</span><span class="token operator">:</span> \\<span class="token punctuation">[</span> 15 <span class="token punctuation">{</span> 16 <span class="token property">"Grantee"</span><span class="token operator">:</span> <span class="token punctuation">{</span> 17 <span class="token property">"Type"</span><span class="token operator">:</span> <span class="token string">"CanonicalUser"</span><span class="token punctuation">,</span> 18 <span class="token property">"DisplayName"</span><span class="token operator">:</span> <span class="token string">"someuser"</span><span class="token punctuation">,</span> 19 <span class="token property">"ID"</span><span class="token operator">:</span> <span class="token string">"8942fa787d23f..."</span> 20 <span class="token punctuation">}</span><span class="token punctuation">,</span> 21 <span class="token property">"Permission"</span><span class="token operator">:</span> <span class="token string">"FULL\\_CONTROL"</span> 22 <span class="token punctuation">}</span> 23 \\<span class="token punctuation">]</span> 24<span class="token punctuation">}</span> 25</code></pre> <p>Watch closely for grants to âAuthenticatedUsersâ and âAllUsersâ in the output and double check to be sure that you intend that access for those buckets. If you do, you should also double check what data is actually in those buckets to make sure sensitive info isnât mixed in with the public info.</p> <p>If you have a lot of buckets, use <code>grep</code>
25 to find any shared with one of those groups.</p> <p>Finally, if thereâs any concern that you may have an object with public permissions inside a bucket that is otherwise private, then youâll want to check the permissions on those object more comprehensively. If you have a lot of objects, this could take awhile.</p> <p>Below is a quick and dirty bash hack to do this for you. It will print out the key for any object with permissions granted to the public or all authenticated users. Youâll need to specify a bucket ID before pasting into a terminal window. If you see any keys printed in the results between the progress dots, youâll want to check those objects out carefully.</p> <pre class="language-bash"><div class="language-label language-label-bash">bash</div><code class="language-bash"> 26<span class="token assign-left variable">BUCKET</span><span class="token operator">=</span><span class="token string">"\\[YOUR BUCKET ID\\]"</span> 27 28<span class="token assign-left variable"><span class="token environment constant">IFS</span></span><span class="token operator">=</span><span class="token variable"><span class="token variable">$(</span><span class="token builtin class-name">echo</span> <span class="token parameter variable">-en</span> <span class="token string">"<span class="token entity" title="\\\\">\\\\</span>n<span class="token entity" title="\\\\">\\\\</span>b"</span><span class="token variable">)</span></span> 29<span class="token keyword">for</span> <span class="token for-or-select variable">key</span> <span class="token keyword">in</span> <span class="token punctuation">\\</span><span class="token variable"><span class="token variable">\`</span>**aws s3api list-objects** <span class="token parameter variable">--bucket</span> <span class="token string">"<span class="token variable">$BUCKET</span>"</span> <span class="token operator">|</span><span class="token function">grep</span> <span class="token string">'"Key"'</span> <span class="token operator">|</span> <span class="token function">sed</span> <span class="token parameter variable">-e</span> <span class="token string">'s/^.\\*"Key": "//'</span> <span class="token parameter variable">-e</span> <span class="token string">'s/",\\\\? \\*$//'</span><span class="token punctuation">\\</span><span class="token variable">\`</span></span> 30 <span class="token keyword">do</span> <span class="token builtin class-name">echo</span> <span class="token parameter variable">-n</span> <span class="token builtin class-name">.</span> 31 **aws s3api get-object-acl** <span class="token parameter variable">--bucket</span> <span class="token string">"<span class="token variable">$BUCKET</span>"</span> <span class="token parameter variable">--key</span> <span class="token string">"<span class="token variable">$key</span>"</span> <span class="token operator">|</span> <span class="token function">egrep</span> <span class="token string">'(AuthenticatedUsers|AllUsers)'</span> <span class="token operator">&&</span> <span class="token builtin class-name">echo</span> <span class="token string">"Public: <span class="token variable">$key</span>"</span> 32<span class="token keyword">done</span> 33</code></pre> <p><strong><em>Note: any interaction with AWS can trigger usage charges, so beware especially if you have a large number of files.</em></strong></p> <h4 id="what-about-encryption"><!>What About Encryption?</h4> <p>The fact that all these folks misconfigured AWS permissions and put sensitive data in public places is bad enough, but for sensitive customer data and even top secret data, <strong>simple permissions should not be considered âgood enough.â</strong></p> <p>Sensitive data should always be encrypted so that even if a server is compromised or a rogue Amazon employee gains access, the data is still unreadable except to authorized users. This seems like basic common sense, but unfortunately, few organizations encrypt their data in this way. None of the recently embarrassed organizations did.</p> <p>If cryptographically backed access controls (regardless of where data is stored) are something that could aid your company, please <!> to see if we can help.</p> <p><strong>UPDATE 8/17/17</strong>: <!>, a voting machine supplier, left sensitive data on 1.8 million Americans in an unsecured AWS S3 bucket.</p> <p><strong>UPDATE 8/31/17</strong>: <!>
33, a provider of backend software for the hospitality industry, left credit card details, contracts, internal financials and more exposed in an unsecured AWS S3 bucket.</p> <p><strong>UPDATE 9/1/17</strong>: <!> exposed 600 gigabytes of data containing 4 million records of Time Warner Cable customersâ sensitive data via yet another public S3 bucket.</p> <p><strong>UPDATE 9/5/17</strong>: <!>, a military contractor, and former partner TalentPen, leaked over 9,000 resumes of law enforcement, military, and government personnel via a public S3 bucket.</p> <p><strong>UPDATE 9/20/17</strong>: <!>, the owner of Parmaount Pictures, Comedy Central, MTV and more media properties, left its data exposed on an unprotected S3 bucket. Among the things found in the data were the keys to the Viacom kingdom including their master AWS keys, GPG keys, and server credentials. A hacker could have used these to steal enormous amounts of data from Viacom. Viacom says there was âno material impactâ from the exposed data.</p> <p><strong>UPDATE 9/22/17</strong>: <!> posted confidential documents and internal user credentials to an unsecured S3 bucket.</p> <p><strong>UPDATE 9/22/17</strong>: <!>, an auto tracking company, leaked hundreds of thousands of records with customer data in an unsecured S3 bucket.</p> <p><strong>UPDATE 11/21/17</strong>: <!> stored US CENTCOM and PACCOM intelligence data scraped from open Internet resources.</p>`,1);function ma(T,z){const x=K(z,["children","$$slots","$$events","$$legacy"]);Y(T,q(()=>x,()=>P,{children:(I,aa)=>{var S=X(),u=n(M(S),6),U=r(u);i(U,{"aria-hidden":"true",tabindex:"-1",href:"#security-by-obscurity",children:(a,l)=>{var s=H();t(a,s)},$$slots:{default:!0}}),e(),o(u);var d=n(u,10),D=r(d);i(D,{"aria-hidden":"true",tabindex:"-1",href:"#amazon-warns",children:(a,l)=>{var s=J();t(a,s)},$$slots:{default:!0}}),e(),o(d);var h=n(d,8),C=r(h);i(C,{"aria-hidden":"true",tabindex:"-1",href:"#auditing-yourbuckets",children:(a,l)=>{var s=Q();t(a,s)},$$slots:{default:!0}}),e(),o(h);var k=n(h,2),W=n(r(k));i(W,{href:"https://aws.amazon.com/cli/",rel:"nofollow",children:(a,l)=>{e();var s=p("Amazonâs command line interface tool");t(a,s)},$$slots:{default:!0}}),e(3),o(k);var m=n(k,24),E=r(m);i(E,{"aria-hidden":"true",tabindex:"-1",href:"#what-about-encryption",children:(a,l)=>{var s=Z();t(a,s)},$$slots:{default:!0}}),e(),o(m);var g=n(m,6),j=n(r(g));i(j,{href:"/contact-us/",children:(a,l)=>{e();var s=p("reach out to IronCore Labs");t(a,s)},$$slots:{default:!0}}),e(),o(g);var b=n(g,2),B=n(r(b),2);i(B,{href:"https://www.theregister.co.uk/2017/08/17/chicago_voter_leak/",rel:"nofollow",children:(a,l)=>{e();var s=p("ES&S");t(a,s)},$$slots:{default:!0}}),e(),o(b);var f=n(b,2),O=n(r(f),2);i(O,{href:"https://www.theregister.co.uk/2017/08/22/open_aws_s3_bucket_leaked_hotel_booking_service_data_says_kromtech/",rel:"nofollow",children:(a,l)=>{e();var s=p("Groupize");t(a,s)},$$slots:{default:!0}}),e(),o(f);var y=n(f,2),N=n(r(y),2);
33i(N,{href:"https://www.cyberscoop.com/time-warner-cable-data-leak-kromtech-aws-s3/",rel:"nofollow",children:(a,l)=>{e();var s=p("Charter Communications");t(a,s)},$$slots:{default:!0}}),e(),o(y);var v=n(y,2),G=n(r(v),2);i(G,{href:"https://threatpost.com/military-contractors-vendor-leaks-resumes-in-misconfigured-aws-s3/127803/",rel:"nofollow",children:(a,l)=>{e();var s=p("Tigerswan");t(a,s)},$$slots:{default:!0}}),e(),o(v);var w=n(v,2),L=n(r(w),2);i(L,{href:"http://thehackernews.com/2017/09/viacom-amazon-server.html",rel:"nofollow",children:(a,l)=>{e();var s=p("Viacom");t(a,s)},$$slots:{default:!0}}),e(),o(w);var $=n(w,2),V=n(r($),2);i(V,{href:"https://mackeepersecurity.com/post/verizon-wireless-employee-exposed-confidential-data-online",rel:"nofollow",children:(a,l)=>{e();var s=p("Verizon Wireless");t(a,s)},$$slots:{default:!0}}),e(),o($);var _=n($,2),R=n(r(_),2);i(R,{href:"http://www.ibtimes.co.uk/over-500000-car-tracking-devices-passwords-accidentally-leaked-due-misconfigured-cloud-server-1640405",rel:"nofollow",children:(a,l)=>{e();var s=p("SVR Tracking");t(a,s)},$$slots:{default:!0}}),e(),o(_);var A=n(_,2),F=n(r(A),2);i(F,{href:"https://www.upguard.com/breaches/cloud-leak-centcom",rel:"nofollow",children:(a,l)=>{e();var s=p("The Pentagon");t(a,s)},$$slots:{default:!0}}),e(),o(A),t(I,S)},$$slots:{default:!0}}))}export{ma as component};
Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.