DFIR

Analytical lifting, how Olympic Weightlifting helped with my IR workflows/analysis

m0rt0n86Leave a comment

Don’t worry I am not going to make this a #dfirfit post with lifts and all of that but wanted to share how my Olympic Weightlifting (which is “technical” in the sense of movements/positioning and mindset) helped strengthen my IR workflows/analysis when I have investigating. Now I am not saying everyone should go lift and they will magically be a better at their job (though it could help with stress and other things, but that is not the point here). Just wanted to share some personal experience and something I noticed after some time off from lifting due to my chronic health issues. Before anyone ask why I lift this way versus just general strength and conditioning it just happens to be a form a movement I enjoy (granted its a love/hate relationship) and I also do enjoy the competition aspect of it as well… Just don’t tell my anxiety that but also at the same time I use it as a tool to work on that

Now why be analytical with this? as someone that is mostly a remote athlete I sort of had to as I do not get direct feed back like I would if I was at my gym lifting (Don’t worry I do record lifts and send them to my coach and do try and make the commute out to the gym when I am feeling good). As I’ve developed a connection with how my body is moving from yoga even before I started lifting so I had a slight advantage per say. You will get to a point when you can feel that your weight is shifted to a pinch foreword or you transitioned from your first pull to your second pull a pinch earlier than normal. A lot of this is just minor variations that happen in a lift and most of the time the faults come from when you first start pulling the bar of the ground and next time you know you have a learning opportunity for your next lift/session. It’s these small little details you need to pay attention to and feel and you work on rep after rep after rep and also doing accessory movements and prehab for imbalances you might discover you have/ares you need some work at.

Ok I bet you might be wondering what the hell any of that means if you don’t lift or how that ties into not even incident response but just general analysis (or to keep it “cyber” a SOC analyst). Well I’ll break it down into nice little bullet points of things that helped me along the way and to help make sense of the things I noted above and take what you want from it. Maybe it might help you realize that hobbies you enjoy could still help you with skills you might need in your “9-5”.

  • First and foremost, You will make mistakes and you will “miss lifts”
    • Tying the miss lifts in you will miss read a log/data set, might think an alert is a false positive, or go down the rabbit hole analyzing trace data/mft and try and follow the “bad thing” only to find that it was just either an IT background script or just the way a peice of software worked and be a false positive after all. This is hard concept as no one likes to be wrong, no one likes to miss when it comes to lifting (or to bomb out at a meet, aka not make any lifts at a meet) or because something was missed a P4/P3 incident turns into a P2/P1. No one is perfect (good luck telling that to use of ADHD/Anxiety…which is a hole other subject). You will make mistakes it is how to take them and if you like them beat you down or take them for what they are, something to learn from and grow. I know back when I was doing engineering the joke always was if you don’t take down prod once, are you even a sys admin/engineer? Well I attempted to patch and update the SIEM at my old job and it was down for a month…talk about a “learning experience”. Now obviously there are situations that can be more dramatic and hit harder (ie ransomware) but feel that shittyness for a bit (because feeling emotions is good as I am learning) but you need to noticed that weakness and make it an area to work on, talk with your manager/leadership about training or going outside of work for mentorship/guidance on it, research it and come back better for it after you get some “reps” in. but for a TL’DR don’t quit because you missed, make it something you can learn from and grow and reconise when you are falling back into bad habbits
  • Successories…aka accessories movements..aka training/research
    • I know I mentioned with the main lifts you are doing rep after rep after rep which gets you strong in those positions and gets you to feel things for your core lifts or core job/skills for job. If there is one thing I have learned from both of these is you can always learn something. Technology is always changing so you need to do what you can (not burn yourself out in the process) but make time if you can for doing accessory’s or trying that can help out with your primary job function even if it is just a different way of doing one of the task. In a weightlifting approach it might be doing clean pulls or snatch deadlifts, it’s not a full snatch or clean but helps us work on those positions we need to get stronger at for core skills. This would also be doing CTFs/Labs with different tools that you might not be used to using. Another example would be hamstring curls, pull ups, farmers carries, or even curls to hit weak spots that will help with the main lifts this might be learning a specific function of a tool better or learning how to write more complex detection’s/queries, or just general researching malware/vulnerabilities that have been affecting $employer. Again the purpose of these are not to burn you out or lead to more fatigued if you are having an off day, pull these back or skip them. Also if you cane get $employeer to help with this, and get leadership support, awesome! I am also not saying to go buy subscriptions to every place too follow the KISS methodology about it and if you noticed an area bring it up and try to work on it when you can. Which also ties into the first point I made about not giving up because you made a mistake or weren’t perfect about it. Before I forget this can also be soft skills too (Lets be real, most of us also need to work on writing reports/documentation…)
  • Communication is key
    • Not just the external communication but also internal…listening to your body and how you are feeling. You might have days were you had some personal/family stuff going on (or in my case chronic health issue) and just are not feeling it, having issues with simple task (which can be frustrating and again lead to point number 1 and why I made it the first point). Talk to your peers/managers/leaders or in a lifting sense your coach. You might need a day or two with pulled back responsibilities or even need to take some mental health/”sick” days to recharge a bit (this also does not make you weak FYI). If you don’t tell anyone what is going on no one will know and can help. You are just going to push though and while it might not lead to a physical injury like with lifting it will likely lead to burnout or just emptying your..umm..f’s to give which leads to mistakes and can start a negative cycle really quick. This also leads to the prehab stuff as if you have something that is nagging its going to make things harder so work on addressing by communication and building a network/support system to help with those nagging issues can get you back on track along with sucesories. Being honest this is still an area I am working to improve on but still wanted to include it
  • Slow is smooth and smooth is fast…positions
    • AKA workflow. Not talking about IR playbooks or anything like that but how you move though your work ie how scope out/triage an alert/event/detection, how your handle endpoint investigations, or log analysis. Some certifications or blogs will tell you a way to do it, much like there are defaults/standard form for lifts but you need to make adjustments as someone that is 6’2″ my start with lifting or position is not going to be the same as someone that is 5’2″ (lets also leave mobility out of its too..) You need to find the methods that work for you and almost make them clock work and get good at feeling for them or the timing in a sense (aka when you need to pivot to another dataset or how you handle investigation). Obviously laws, regulation, or policy might dictate the end results (ie how things need to be noted on evidence/chain of custody, templates for reports) but making the bulk of it work for you and your mind works and likes to “move” or handle things will only aid you and make the bulk of the work flow better and lead to less jumping around and help with interpreting/automating things which can also help if you actually do have the ability to build out automation via playbooks with SOAR, Jupyter notebooks or other functionality like that

I was going to make this a bullet but since it ties into the first bullet with networking/mentorship and communication but getting a coach versus trying to do it all on your own can help with. Also try and have fun…I know some days are going to suck and be hard but try to enjoy it the best you can. This isn’t your life it is just a piece of it. I know there is the push of grind culture and if that works for you cool. But personality I know when I am on my death bed or when my time comes I would rather be remember as someone that did the work and had some passion but was still human and a decent person versus no one being there but hey they caught an alert that stopped APT 29 or only lived to lift..

Take from this what you can I know some of the lifting analogy might not fully work for some but really this has been on my mind for a little while now and felt like writing about it. I’m sure I might need to make some updates down the road but this was a lot more mentally fatiguing than I thought it would be. Maybe I should of gotten on my soap box and ranted and saved this for a rainy day, but I found it interesting how hobbies and other things helped my in my career development and might help others tie in hobbies and other activities and see how the lessons learned can help build in other areas.

IR, what I wish I knew a year ago

m0rt0n86Leave a comment

I know it has almost been a year since I posted anything in this thing, but with COVID pandemic and also starting a job were I am now in IR (Incident Response). I spent this time trying to learn new processes,procedures, and tools used at $employer. I tried to tell myself when I learned something to post about it, but it was just followed up with something else. So instead of that I figured I’d reflect on the almost year now in my current role and with a “if I could do it again” what do I wish I spent some extra time learning. While of course having a good understanding of the broad levels of Incident response (like having a base understanding of the steps of NIST (NIST 800-61) and some basic understanding of what to look for in logs/SIEM alerts was helpful there were a few other things I needed to work on to build up better base, which I’ll list below.

  • Logs-Don’t focus on a specific log source, be knowledgeable with a few different logs sources and how to read them. I know at the old gig I had more Firewall logs than I could shake a stick at so I wasn’t really sure how to read/fully interpret some of the other logs. When I first started here my auto pilot response was to look at the firewalls when I was going after my first “incident”. Boy was that a mistake. Not all firewall logs are created equally (some could be stateful versus next gen). Some situations these could be useful but for me a better base was understanding Proxy, VPN, Windows(Authentication, DNS and of course events), EDR to better tell the story. Which your SIEM might be able to do , but depending on the amount of logs hitting it and events/alerts you might be better off looking at log aggregation (like ELK) to really filter and get into the nitty gritty of what you are looking for.
  • Linux/SQL/ELK Command line/searching-While I could do basic navigation on the command line, I couldn’t grep/less and use switches for the life of me. Still learning and getting better with switches and some piping, this can also be sql from the command line as well. I use grep a lot when looking at endpoint data and log files from linux host so not being afraid to try different switches/pipes and using grep/less to search for certain things along with understanding how to search for data in telemetry that was an sql database (and can be useful also for things that use osquery…looking at you open SOC CTF). Also I have some basic stuff with Elk and command line now, still need to work on regex for even better filtering/searches to get rid of some of the noise.
  • Net flows/PCAPS– Grouping these two together as I generally grab PCAPs from specific net flows. While net flows were an issue at old gig, we use flows a lot with NSM, being able to read the data that the flows are giving you along with deciding if it is worth pulling the pcap from the NSM sensor to get information about an incident. I had done work with pcaps before from a network troubleshooting standpoint and light IR but taking advantage of looking at the TCP/HTTP streams to see what is really going on with the traffic versus what proxy/firewall’s are saying in terms of the communications. Also being able to look at the HTTP headers to see what might be going on and also knowing how to pull data from the HTTP streams (useful to pull downloads that might of came in from IoC’s or C2’s for further analysis/sandboxing).
  • Report Writing/timelines-While everyone can write notes and other things (if not, you should really be keeping your own notes outside of Incident tracking/SOAR/ticketing system’s incase those are unavailable in a critical incident. You can also quickly look back at them for repeat offenders for what to look for). However, being able to write something that is more in-depth and technical versus an executive summary is something that takes work. Summary’s aren’t going to need the same level of depth as a full technical report. Also keeping notes on the timeline of activity is also good to have in order to help build timelines (since they can help with the summaries as well).
  • Understanding dynamic analysis outputs from sandbox tools– Was tempted to call this system internals but figured this made more since as I know about LOLBINS and .net assembly getting dumped into memory but understanding the connections of what is going on and the associated actions and being able to explain that in both technical/non-technical way (again, the writing thing). Along with being able to tie that information back into the the endpoint data/and searches to help find artifacts and some of the technical happenings on the host.
  • Scripting– This is still a weakness of mine (mostly regex) and something I’m still working on as it is a “use it or lose it” skill. Since I’m not scripting a lot of the time my limited abilities go right out the window a lot of the time. Still working on my python and Powershell to a lesser extent to help get some of the easy info I need for stuff to get pulled.

There are just a few of my key things I’ve noticed with myself. Above all else though is not being afraid to make mistakes. I know with IR work you want to do it right 100% of the time and want to keep the $org out of the news, but if you are afraid to try things you will not learn anything and become better. Same goes for trying tools/technics find what works for you and is in scope with your org and just do your job to the best of your abilities. Also do not be afraid to network and ask question’s inside/outside your org.

These are my keypoints, I’m sure there are others who use different tools/technics/procedures but this is what I’ve really learned over the past year. I know I still have more to go, but that goes with the IR territory. Keep learning, Keeping trying to find the 5 “w”‘s with what you know or are trying to learn. Ok so this actually the most important-use your PTO/Take mental health days when needed. Your no good to your org if you are always burnt-out and depressed so take care of yourself!

So long and thanks for the Response: why are we skipping post incident activities

m0rt0n86Leave a comment

While re-going though some Incident response topics with Security Blue Team, Cybary and LinkedIn Learning there was a common thread that they all had if they were going though NIST 800-61 (or even SANS standards) was NOT to skip the post incident activities. business is back up right? who cares? we got it all solved so post incident activities means going out to the local watering hole and having an adult beverage or six. That is what you mean right? No. not at all (although if you wanted to post post incident activities I guess this could work)

Now your might be thinking why well if your are a fan of standards/frameworks (If you are in Cyber you probably are since you do have compliance you need to meet among other things) there is this hole step that can also improve the process for next time, as it is going to happen as someone is going to have an off day and click yet another link to make your life full of fun. Also you wouldn’t hire a Pen-testing firm to come in and not review the findings and look for ways to improve the organizations security posture? You wouldn’t do Red team/purple team engagement with tools like Red Canary’s “Atomic Red Team” or MITRE Caldera would you? If not what are you really getting out of it other than doing a task to check the box for something which is not going to do anything to help you or the org (except maybe if you want to give yourself a gold star). You use these tools and skills to look at oversights, detection/alerts that are missing or not as clear. Which you would work to fix and to help your cause you could map this to ATT&CK/Shield (as we all know leaderships likes charts and frameworks are super helpful for some).

So the question is, we make time for that? why are we not making time for the post incident review? I get it. its another round table, another day potently spent going over process/procedures but if it makes your security posture better by looking at the things you would with a Pen-Test or purple teaming, why skip out on it. It’s like stomping out a fire without making sure it’s embers are fully out. In this case you don’t improve the response to a particular incident guess what happens? those embers (incident) rises into a flame and next thing you know you are dealing with a similar incident responding in the same way. Did that really work? sure you put out the big fire, but you did not ensure it was fully out because you are still getting hit in the same way you did before. now if your org is not giving you the time or capabilities to fully put out the fire, you probably do not have full management buy in into your incident response program as that should be the priority for those on the Computer Emergency Repose Team (CERT/Cyber Security Incident Response Team (CSIRT) when facing an incident.

I do understand that most are working with limited business and that security is just a coast center (unless you are an MSSP or something among those lines). but how can the business really function under normal operations if you still have kindling flames of incidents (attack vectors that your IR plan/policy lacks coverage for) if you are constantly wasting time going back to “remediating” and leaving it at that. Maybe its the military and having to instruct a few classes that thought me the importance of After Action Reviews (AAR) but not doing that security incidents and then wondering why you are dealing with the same attacks over and over again fits the definition of insanity (doing the same thing over again and expecting a different result). Doubly so if the business gets effected by this time after time, you would think they would want it to be prevented to prevent any bad publicity and having to issues statements over and over again.

Taking the time to try and close the gap with post incident activities ensure senior leadership understand that this is apart of the process and ensure that it is the plan/policy. if they do not want to listen and want you to move on to other things remind them of their buy in/enforcement. If not, use ethics committees and other tools to ensure policies/plans/procedures are being enforced and allowed to be followed though to the end, the very end. At the end of the day this is to better the security posture to allow an org to keep operations up by having plans in place to avoid shutting down production due to a cyber incident that effects the CIA triad.

I know not every org is like this, but when going though videos and other training the theme that always seems to be around is “DO NOT SKIP THE POST INCIDENT ACTIVITIES” there is defiantly something up with the industry and something we need ensure we are doing. Along with doing IR exercises (if you want some gamification of IR tabletops look at Black Hills Information Security Backdoors and Breaches card game) But I’ll save that for another time.

DEF CON 28-DEF CON Safe Mode

m0rt0n86Leave a comment
DEF CON needed to Run some A/V…Image (c) © DEF CON

First DEF CON, first full Virtual conference I attended. This post is going to be in different parts with with the overall experience, the things I really liked and kind of an simple overview of the OpenSOC CTF (which I thoroughly enjoyed). I am still trying to get caught up on some of the varies talks from the other villages as a lot of my time was spent working on the OpenSOC CTF and the various workshops/panels with Blue Team Village (BTV). I was hoping to do a better write up but life found away and I wanted to try and spend some extra time with my wife since me being “Out” due to DEF CON put some extra strain on her

What I really liked?:

  • OpenSOC CTF: This was my first time doing the OpenSOC CTF and was not sure what I was fully getting myself into as I was used to the other jeopardy and other styled more red team focused CTFS with web-apps to break, Stenography, and some Forensics challenges but what really captured my interest was this was more like a full fledged SOC environment and you had to investigate several different incidents. The tools I have never really used before or interrupted (PF sense, Thinkst Canary tokens, OSQuery, Suricata, Greylog, Snort, Velociraptor, and Moloch). Thankfully the Blue Team Village had some early “workshops” showcases the tools and how to use them. While learning the syntax of each of the different tools (thankfully Greylog is built on top of an ELK which I do know some of the Syntax for) But once I learned the syntax and got a better idea of what I was looking for it I was able to do good on following the trail of several challenges. Which they did lead on each new “flag” was just adding another piece of the puzzle and told the story of what you were looking at.
    The networking portion of looking at the events/logs from firewalls/IDS/email I did really well with (along with packets) but once we got into the nitty gritty of filtering though Sysmon (thanks to WinLogbeats) and other endpoints I started to falter a little, I know its an area I am currently working to improve on with DetectionLab. I was able to figure some events out (like power shell script running and what script was ran and was able to pull some “IOC’s” but what made this not fun, was trying to bounce between this, the workshops I signed up for and the various talks/panels…and you know eating XD (I know it was DEF CON but I’m not about that life).

    I was not expecting to win or even go top 10 doing it solo but for my first time using the tools and finishing at the lower end of the 50th precedential or so I’m pretty pleased with my self and I got some weak spots I need to work on and that is perfectly ok!
  • Workshops: While there was some bad as it was a FireHose at some points (and one being around 10PM EST) but these really were the bread and butter other than the OpenSOC CTF. Every since I did a splunk workshop at GrrCON last year (it was really a guided Boss of the SOC v2 AKA BOTS v2) I try to get into workshops. The ones I really liked were creating Jypter notebooks playbooks for threat hunting, NSM in the cloud (deploying Securiyt Onion on AWS..noice). and also writing Yara rules. I know I need to look back on the Yara rules and Jyper notebooks as A) I was really tired and B) in the middle of a good flow with the OpenSOC CTF XD.
    Both are tools I want to get better at using/understanding to help automate some things and to build up better rule sets (in the case of Yara). There was also one with IR which was good, but it was like a CTF and I’ll guide you (which I need to find the link for as it is always up) Thankfull the slides were shared to attendees so now that things are calming down a little a gain its time to go though them again (doubly so once I finish my Detection Lab set up as I also want to add RITA from Active Countermeasures and maybe a metasplotable or OSWAP Juice box vulnerable web app) I can start really applying these and playing around with them. That is the con that if you don’t use it right away, you lose it and this was really the case as back to the normal grind of $employer I really didn’t get a chance to work with these as catching up and other stuff I didn’t get the chance to review/play around like I wanted. If this wouldn’t of been so delayed it would of helped too XD
  • Talks/Panels: Part of the reason why It’s taken me so long to write this is I am still trying to catch up on talks from all the villages. I know being from the Metro Detroit area I spent some time getting some entry knowledge with automotive hacking and some of the basics of automotive networks which I was clueless about, it was good that it was pretty cut and dry simple and wasn’t overly complicated. I really did enjoy the panel talk on IR. As someone that has a rapidly growing interest in DFIR/Threat Hunting/NSM and more active defense (Decoys/Honeypots) this was really informative to listen too from people that do that for a living and get some insight. Another good talk was about BlueSpawn It was funny we just worked with this with on my intro to securtiy course with John Strand/Black Hills Information Security. It was neat to get more background on the tool and part of the reason for its development. If you don’t know Blue Spawn is an open source active defense/EDR product based in Powershell (at lest for what we used it for with the course).

not bad in itself, but made it hard

  • Discord…It was nice for all the Villages and DefCON to have seperate channels. however, the alerting was so distracting with people posting in all the channels. I tried to mute were I could, but then I’d miss things or just ignore channels all together (Sorry Red Team Village). Also I’m “that guy” that has the taskbar on top set to autohide (its clutter) so everytime there was a new ping start menu would drop down and just be distracting. Granted I know this is small and really it was nice to have it “break out” like that and now they are live communities still even after DefCon. But for someone that gets distracted pretty easy or annoyed with things, this was a big no from me XD.

I know this was long over due but with work and trying to take care of some projects around the house due to DEF CON and weather that needed to be taken care of. This was probably the only way I’d be able to attended a DEF CON as I’m not sure the wife would be thrilled with me taking PTO to go fly to Vegas to attended this. Maybe if I can talk to my $employer/contract house and see if they can cover some of it. Maybe life might find away and I get head out that way and maybe score a badge (or a BTV one as well)

On-words to Wild West Hackin’ Fest in a few weeks!