Security information and event management

pledge info and tied(p)t warinessIntroduction guarantor selective information and solution Management (SIEM) automates incident identification and resolution based on streng thened in melodic phrase rules to jock improve configuration and alert lag to critical intrusions. IT audits, standards and regulatory wantments have now become an important fraction of to the highest degree enterprises day-to-day responsibilities. As part of that burden, organizations ar spending substantial quantify and energy scrutinizing their gage and event poundarithms to track which dusts have been accessed, by whom, what activity took place and whether it was appropriate. Organizations be increasingly human faceing towards information-goaded automation to help ease the burden. As a result, the SIEM has taken form and has stomachd focused solutions to the problem. The protective c all over information and event focal flower market is driven by an extremely increasing quest for c ustomers to meet conformation requirements as well as continued necessity for concrete- age awareness of foreign and internal holy terrors. Customers posit to analyze certificate event data in palpable time (for affright charge) and to analyze and typography on logarithm data and chiefly this has made aegis system information and event focussing market much(prenominal) demanding. The market sojourns fragmented, with no dominant vendor.This report entitled Security training and solution Management (SIEM) Solutions gives a clear view of the SIEM solutions and whether they tooshie help to improve intrusion espial and response. Following this introduction is the primer coat section which deeply analyzes the evolution of the SIEM, its architecture, its relationship with the log management and the get hold of for SIEM carrefours. In the epitome section, I have analyzed the SIEM functions in percentage point along with real world examples. Finall(a)y the conc lusion section summarizes the paper. sceneWhat is SIEM?Security Information and Event Management solutions are a combination of two dissimilar products namely, SIM (security information management) and SEM (security event management). SIEM technology provides real-time synopsis of security alerts generated by network hardware and applications. The objective of SIEM is to help companies serve to attacks faster and to organize push-down storages of log data. SIEM solutions come as software, appliances or managed services. Increasingly, SIEM solutions are being used to log security data and generate reports for compliance purposes. Though Security Information and Event Management and log management tools have been complementary for years, the technologies are expected to merge.Evolution of SIEMSIEM emerged as companies free-base themselves spending a lot of m peerlessy on intrusion light uponion/prevention systems (IDS/IPS). These systems were helpful in detecting external attac ks, exclusively because of the merged trust on mite-based engines, a tumid number of false positives were generated. The commencement-generation SIEM technology was designed to condense this signal-to-noise ratio and helped to capture the most critical external threats. Using rule-based correlation, SIEM helped IT detect real attacks by foc use on a subset of firewall and IDS/IPS events that were in violation of policy. Traditionally, SIEM solutions have been expensive and time-intensive to maintain and tweak, but they cypher the big headache of sorting finished excessive false alerts and they efficaciously protect companies from external threats. While that was a timber in the well(p) direction, the world got to a greater extent complicated when new regulations such as the Sarbanes-Oxley lick and the Payment Card Industry Data Security Standard followed much stricter internal IT controls and assessment. To satisfy these requirements, organizations are required to suck in, analyze, report on and get by in all logs to monitor activities inside their IT infrastructures. The idea is non solo to detect external threats, but overly to provide periodic reports of user activities and defecate forensics reports surrounding a given incident. Though SIEM technologies collect logs, they bear upon only a subset of data related to security breaches. They werent designed to take the right-down volume of log data generated from all IT comp matchlessnts, such as applications, switches, routers, databases, firewalls, operational systems, IDS/IPS and Web proxies. With an idea to monitor user activities rather than external threats, log management entered the market as a technology with architecture to handle much larger volumes of data and with the ability to extend to meet the demands of the largest enterprises. Companies employ log management and SIEM solutions to satisfy different employment requirements, and they have to a fault light upon out tha t the two technologies work well together. Log management tools are designed to collect report and archive a large volume and breadth of log data, w here(predicate)as SIEM solutions are designed to correlated a subset of log data to point out the most critical security events. On looking at an enterprise IT arsenal, it is possible to see both(prenominal) log management and SIEM. Log management tools often assume the exercise of a log data warehouse that filters and forwards the necessary log data to SIEM solutions for correlation. This combination helps in optimizing the return on investment magic spell also reducing the cost for implementing SIEM. In these tough economic time it is likely to see IT trying to stretch its logging technologies to solve even more problems. It leave al ace expect its log management and SIEM technologies to work impending together and reduce overlapping functionalities. Relation between SIEM and log management Like umpteen affaires in the IT i ndustry, theres a lot of market status and buzz coming around regarding how the original term of SIM (Security Information Management), the succeeding marketing term SEM (Security Event Management), the newer combined term of SIEM (Security Information and Event Management) relate to the long standing process of log management. The basics of log management are not new. Operating systems, spins and applications all generate logs of almost sort that contain system-specific events and notifications. The information in logs whitethorn vary in overall usefulness, but before oneness freighter derive much value out of them, they first need to be enabled, then transported and finally stored. in that respectfore the way that one does gather this data from an often distributed escape of systems and get it into a centralized (or at least semi-centralized) location is the first challenge of log management that counts. There are varying techniques to accomplish centralization, ranging from standardizing on the syslog mechanism and then deploying centralized syslog servers, to using commercial products to verbalize the log data acquisition, transport and storage issues. Some of the other issues in log management include working around network bottlenecks, establishing tried event transport (such as syslog over UDP), setting requirements around encryption, and managing the black data storage issues. So the first steps in this process are figuring out what type of log and event information is in need to gather, how to transport it, and where to store it. But that leads to another major(ip) consideration rough what should one person want to do with all those data. It is at this point where the basic log management ends and the higher-level functions associated with SIEM begins. SIEM products typically provide m any of the features that remain essential for log management but issue event-reduction, alerting and real-time analysis capabilities. They provide the stratum of technology that allows one to say with confidence that not only are logs being gather but they are also being reviewed. SIEM also allows for the importation of data that isnt necessarily event-driven (such as vulnerability s rousening reports) and it is enjoyn as the Information portion of SIEM.SIEM architectureLong term log management and forensic queries need a database built for capacity, with file management and narrowion tools. wretched term threat analysis and correlation need real time data, CPU and RAM. The solution for this is as followsSplit the feeds to two concurrent engines.Optimize one for real time and storage up to 30 days of data. (100-300GB)Optimize the second for log compression, retention, and query functions. (1TB+)The block diagram showing the architecture of the SIEM is as follows Source reference 2A collector is a process that gathers data. Collectors are produced in many shapes and sizes from agents that run on the monitored device, to centra lized logging devices with pre-processors to split stream the data. These bottom be simple REGEX file parsing applications, or mingled agents for OPSEC, LEA, for .Net/WMI, SDEE/RDEP, or ODBC/SQL queries. non all security devices are kind enough to forward data, and triple arousal methods, including active pull capabilities, are precise essential. Also, since SYSLOG data is not encrypted, it may need a collector to provide encrypted transport.A threat analysis engine leave need to run in real time, continuously processing and correlating events of interest passed to it by the collector, and inform to a sympathize with or exhi goation layer application about the threats found. Typically describe events that has happened for 30 days are sufficient for operational considerations. A log manager lead need to store a great lot of data, and may take either raw logs or filtered events of interest, and need to compress store and index the data for long term forensic analysis and compliance reporting. Capacity for 18 months or more of data is likely to be required. Year end closing of books and the arrival of the auditors often pick out the need for 12 months of historic data plus padding of several(prenominal) months plot of ground books are finalized and an audit to be completed.At the presentation layer a console forget present the events to the security ply and managers. This is the principal(a) interface to the system for day to day trading operations, and should efficiently prioritize and present the events with a full history and correlation rationale.SIEM functionsWith many subtle differences, there are four major functions of SIEM solutions. They are as follows1. Log desegregation centralized logging to a server2. Threat Correlation the bleached intelligence used to sort through multiple logs and log entries to call attackers3. Incident Management workflow What happens once a threat is identify? (link from identification to containment and eradication). Notification email, pagers, informs to enterprise managers (MOM, HP Openview) Trouble Ticket invention Automated responses execution of scripts (instrumentation) Response and Remediation logging4. Reporting functional Efficiency/Effectiveness Compliance / SOX, HIPPA, FISMA. Ad Hoc / Forensic InvestigationsComing to the business case for SIEM, all leads are perpetually drawn to new technology, but purchasing decisions should by necessity be based on need and practicality. Even though the functions provided by SIEM are impressive they must be chosen only if they fit an enterprises needs. Why use a SIEM?There are two branches on the SIEM tree namely, operational efficiency and strongness, and log management/compliance. Both are achievable with a grave SIEM tool. yet since there is a large body of work on log management, and compliance has multiple branches, this coursework will focus only on using a SIEM tool effectively to point out the real attackers, and t he scald threats to improve security operations efficiency and effectiveness. It merchantman be believed that the most compelling reason for a SIEM tool from an operational perspective is to reduce the number of security events on any given day to a manageable, actionable list, and to automate analysis such that real attacks and intruders asshole be discerned. As a whole, the number of IT professionals, and security focused individuals at any given company has decreased relative to the complexity and capabilities demanded by an increasingly inter networked web. While one solution may have stacks of highly skilled security engineers on staff pouring through individual event logs to identify threats, SIEM attempts to automate that process and can execute a legitimate reduction of 99.9+% of security event data while it actually increases the effective detection over traditional human driven monitoring. This is why SIEM is preferred by most of the companies.Reasons to use a SIEMTo know the need for a SIEM tool in an organization is very important. A defense in depth strategy (industry best practice) utilizes multiple devices Firewalls, IDS, AV, AAA, VPN, drug user Events LDAP/NDS/NIS/X.500, Operating System Logs which can easily generate hundreds of thousands of events per day, in some cases, even millions. No matter how neat a security engineer is, about 1,000 events per day is a practical maximum that a security engineer is about to deal with. So if the security team is to remain tiny they will need to be equipped with a good SIEM tool. No matter how good an individual device is, if not monitored and correlated, each device can be bypassed individually, and the total security capabilities of a system will not exceed its weakest link. When monitored as a whole, with cross device correlation, each device will signal an alert as it is attacked cosmetic surgery awareness and threat indications at each point allowing for additional defences to be brought i nto play, and incident response proportional to the total threat. Even some of the small and medium businesses with that a few devices are seeing over 100,000 events per day. This has become usual in most of the companies says the internet.Real world examples infra are event and threat alert numbers from two different sites currently running with 99.xx% correlation efficiency on over 100,000 events per day, among which one industry expert referred to as amateur level, stating that 99.99 or 99.999+% efficiency on well in excess of 1,000,000 events per day is more common.Manufacturing Company telephone exchange USA 24 hour average, un-tuned SIEM day of deploymentAlarms Generated 3722CorrelationEfficiency 99.06% critical / MajorLevel Alerts 170Effective Efficiency 99.96% Source Reference 2In this case, using a SIEM allows the companys security team (2 people in an IT staff of 5), to respond to 170 critical and major alerts per day (likely to decrease as the worst offenders are firew alled out, and the worst offenses dealt with), rather than nearly 400,000.Financial go Organization 94,600 events 153 actionable alerts 99.83% reduction. Source Reference 2The company above deals with a very large volume of financial transactions, and a missed threat can mean real monetary losses.With respect to the Business Case, a good SIEM tool can provide the analytics, and the knowledge of a good security engineer can be automated and fictionaliseed against a mountain of events from a range of devices. Instead of 1,000 events per day, an engineer with a SIEM tool can handle 100,000 events per day (or more). And a SIEM does not leave at night, find another job, take a break or take vacations. It will be working always.SIEM Selection CriteriaThe first thing one should look at is the goal. (i.e.) what should the SIEM do for them. If you just need log management then make the vendor can import data from ALL of the in stock(predicate) log sources. Not all events are sent via S YSLOG. Some may be sent throughCheckpoint LEACisco IDS RDEP/SDEE encryption picture Scanner Databases Nessus, Eeye, ISSAS/400 Mainframes flat filesDatabases ODBC/SQL queriesMicrosoft .Net/WMIConsider a product that has a defined data collection process that can pull data (queries, retrieve files, WMI api calls), as well as accept input sent to it. And it is essential to be aware that logs, standards, and formats change, several (but not all), vendors can adapt by parsing files with REGEX and importing if one can get them a file. However log management itself is not usually an end goal. It matters about for what purpose these logs are used for. They may be used for threat identification, compliance reporting or forensics. It is also essential to know whether the data captured is in real-time. If threat identification is the primary goal, 99+% correlation/integration/aggregation is easily achievable, and when properly tuned, 99.99+% efficiency is within reach (1-10 actionable th reat alerts / 100,000 events).If compliance reporting is the primary goal, then consider what regulations one is subject to. Frequently a company is subject to multiple compliance requirements. Consider a fortune 500 company like general Electrics. As a publicly traded company GE is subject to SOX, as a vendor of medical equipment and software they are subject to HIPPA, as a vendor to the Department of Defense, they are subject to FISMA. In point of fact, GE must produce compliance reports for at least one corporate division for nearly each and every regulation. Two drawing notes on compliance, and one should look at architecture Beware of vendors with tin reports. While they may be very appealing, and sound like a solution, valid compliance and auditing is about matching output to ones stated policies, and must be customized to match each companys published policies. Any SIEM that can collect all of the required data, meet ISO 177999, and provide timely monitoring can be used to aid in compliance. Compliance is a complex issue with many management, and financial process requirements, it is not just a function or report IT can provide.Advanced SIEM TopicsRisk ground Correlation / Risk ProfilingCorrelation based on gamble can dramatically reduce the number of rules required for effective threat identification. The threat and target profiles do most of the work. If the attacks are risk profiled, deuce-ace relatively simple correlation rules can identify 99%+ of the attacks. They are as followsIP Attacker repeat offendersIP Target repeat targetsVulnerability Scan + IDS Signature match Single Packet of condemnRisk Based Threat Identification is one of the more effective and interesting correlation methods, but has several requirementsA Metabase of Signatures Cisco calls the attack X, ISS calls it Y, wench calls it Z Cross Reference the dataRequires automated method to lapse up to date.Threats must be compiled and threat weightings applied to each sign ature/event. Reconnaissance events are low weighting but aggregate and report on the persistent (low and slow) attacker Finger Printing a bit more specific, a bit higher weighting Failed User Login events a medium weighting, could be an unauthorized attempt to access a resource, or a forgotten password. Buffer Overflows, Worms and Viruses -high weighting -potentially destructive events one need to respond to unless one has already patched/protected the system.The ability to take heed or adjust to ones network Input or auto-discover which systems, are business critical vs. which are peripherals, desktops, and non-essentialRisk Profiling Proper application of trust weightings to reporting devices (NIST 800-42 best practice), can also help to lower cry wildcat issues with current security managementNext-generation SIEM and log managementOne demesne where the tools can provide the most needed help is in compliance. Corporations increasingly face the challenge of staying accountab le to customers, employees and shareholders, and that means protecting IT infrastructure, customer and corporate data, and complying with rules and regulations as defined by the government and industry. restrictive compliance is here to stay, and under the Obama administration, corporate accountability requirements are likely to grow. Log management and SIEM correlation technologies can work together to provide more universal views to help companies satisfy their regulatory compliance requirements, make their IT and business processes more efficient and reduce management and technology costs in the process. IT organizations also will expect log management and intelligence technologies to provide more value to business activity monitoring and business intelligence. Though SIEM will continue to capture security-related data, its correlation engine can be re-appropriated to correlate business processes and monitor internal events related to performance, uptime, capability utilization and service-level management. We will see the combined solutions provide deeper insight into not just IT operations but also business processes. For example, we can monitor business processes from step A to Z and, if a step gets missed, well see where and when. In short, by integrating SIEM and log management, it is easy to see how companies can notwithstanding by de-duplicating efforts and functionality. The functions of collecting, archiving, indexing and correlating log data can be collapsed. That will also lead to savings in the resources required and in the guardianship of the tools. CONCLUSION SIEM is a complex technology, and the market segment remains in flux. SIEM solutions require a high level of technical expertise and SIEM vendors require extensive partner training and certification. SIEM gets more exciting when one can apply log-based activity data and security-event-inspired correlation to other business problems. Regulatory compliance, business activity monitoring and business intelligence are just the tip of the iceberg. Leading-edge customers are already using the tools to increase visibility and the security of composite Web 2.0 applications, cloud-based services and mobile devices. The key is to start with a central record of user and system activity and build an extend architecture that lets different business users access the information to solve different business problems. So there is no doubt in SIEM solutions help the intrusion detection and response to improve.References1. Nicolett.M., Williams.A.T., Proctor.P.E. (2006) Magic Quadrant for Security Information and Event Management, 1H06 RA3 1192006.2. Swift.D. (2006) A Practical Application of SIM/SEM/SIEM Automating Threat Identification3. SIEM A Market Snapshot (2007) from http//www.crn.com/security/197002909jsessionid=BVQXTH11HH14JQE1GHPSKH4ATMY32JVN Date Accessed 20th November,2009.4. WHAT IS SIEM (2008) from http//www.exploresiem.com/resource-center.html Date Accessed 24th No vember, 2009.5. Securing and Managing Your Enterprise An incorporate Approach (2008) fromhttp//www.exploresiem.com/images/WP-Securing-and-Managing-Your-Enterprise.pdf Date Accessed 25th November, 2009.6. Shipley .G.(2008) Are SIEM and log management the same thing? from http//www.networkworld.com/reviews/2008/063008-test-siem-log-integration.html Date Accessed twenty-sixth November, 20097. Levin.D. (2009) The convergence of SIEM and log management from http//www.networkworld.com/news/tech/2009/031909-tech-update.html Date Accessed 26th November, 2009