ReZaAdineH

To estimate the capacity required for log management, you need to determine the EPS and then calculate the amount of disk space needed to store the logs generated at that rate. Here’s a step-by-step guide:

Step 1: Determine the EPS To determine the EPS, you need to know the number of events generated per second. For example, if you have a system that generates 1000 log events per second, then the EPS for that system is 1000.

EPS × DAY = EPD (For example, 1000 EPS x 86,400 seconds = 86,400,000 EPD or 86.4 MEPD)

Step 2: Calculate the size of the log events You need to know the size of the log events to calculate how much disk space will be required to store them. There are two sizes to consider: raw event size and normalized event size.

• Raw event size: Typically, a raw log event is about 600 bytes in size.
• Normalized event size: A normalized log event is usually about 1500 bytes in size.

EPD X RAW = SIZE (RAW) (For example, 86.4 MEPD x 600 = 51,840,000,000 bytes)

EPD X NORM = SIZE (NORM) (For example, 86.4 MEPD x 1500 = 129,600,000,000 bytes)

Step 3: Compress the log events To reduce the amount of disk space required to store the logs, you can compress them. Assuming a compression ratio of 10:1, you can divide the maximum daily size allocation for events by 10 to get the approximate daily disk requirement.

SIZE / COMPRESS = DISK (RAW or NORM) (For example, 52 GB / 10 = 5,184,000,000 bytes for RAW or 129 GB / 10 = 12,960,000,000 bytes for NORM)

Step 4: Calculate the annual disk space requirement Finally, you need to calculate the amount of disk space required for log storage over a year. To do this, multiply the daily disk requirement by 365.

DISK (RAW) × 365 = YEAR (RAW) (For example, 5,184,000,000 x 365 = 1,892,160,000,000 or 1.8 Terabytes for RAW)

DISK (NORM) × 365 = YEAR (NORM) (For example, 12,960,000,000 x 365 = 4,730,400,000,000 or 4.7 Terabytes for NORM)

Step 5: Estimate the total average EPS To estimate the total average EPS for all devices, you need to determine the EPS for each device and then add them together. For example, if you have 10 devices with EPS of 1000, 2000, 500, 1500, 200, 1000, 800, 1200, 900, and 700, the total average EPS would be:

(1000 + 2000 + 500 + 1500 + 200 + 1000 + 800 + 1200 + 900 + 700) / 10 = 1070

Step 6: Calculate the storage requirement for the estimated total average EPS To calculate the storage requirement for the estimated total average EPS, use the following formula:

Step 6: Calculate the storage requirement for the estimated total average EPS To calculate the storage requirement for the estimated total average EPS, use the following formula:

EPD * RAW / 10 * 365 = YEAR (compressed)

Where:

• EPD: Events per day for the estimated total average EPS
• RAW: Raw event size in bytes
• 10:1 is the compression ratio
• 365: Number of days in a year
• YEAR (compressed): The annual compressed storage requirement in bytes

For example, let’s say the estimated total average EPS is 500,000. Using the formula above with the raw event size of 600 bytes, we get:

500,000 * 600 / 10 * 365 = 10,950,000,000 bytes or approximately 10.2 terabytes per year (compressed)

This means that if we want to retain logs for one year, we would need approximately 10.2 terabytes of storage space.

It’s important to note that this is just an estimate and the actual storage requirement may vary based on factors such as retention policies, log rotation settings, and the actual compression ratio achieved.

Step 7: Monitor and adjust Once the log management solution is in place, it’s important to monitor the storage usage regularly and adjust the storage capacity as needed. If the actual storage usage exceeds the estimated capacity, additional storage may need to be added or retention policies may need to be adjusted.

In addition to monitoring storage usage, it’s also important to monitor the EPS and adjust the log management solution as needed to ensure that it can handle the volume of log data being generated.

Conclusion In summary, log management is an important part of any IT infrastructure and requires careful planning and capacity estimation to ensure that the log data can be retained and analyzed as needed. By following the steps outlined in this guide, organizations can estimate the storage capacity needed for log management and adjust as needed to ensure that they have enough storage space to retain log data for their desired retention period.