How to relieve the problem of high memory usage on a switch

How to relieve the problem of high memory usage on a switch

1. Checking Whether High Memory Usage Occurs

Run the display memory-usage command to check memory usage.

<HUAWEI> display memory-usage                                     
Memory utilization statistics at 2016-09-01 10:21:17-03:00       
System Total Memory Is: 867837916 bytes                          
Total Memory Used Is: 203704524 bytes                            
Memory Using Percentage Is: 23%

If memory usage of the S2300, S3300, S2700, or S3700 continuously increases and exceeds 85%, high memory usage occurs on the switch.
If memory usage of the S5300, S6300, S5700, S6700, or a modular switch continuously increases and exceeds 60%, high memory usage occurs on the switch.
NOTE:
Save all the operation records.

2. Checking Whether Memory Usage Continuously Increases

To check whether memory usage continuously increases, run the display memory-usage command and collect memory usage information periodically, for example, once every hour or day.

<HUAWEI> display memory-usage                                     
Memory utilization statistics at 2016-09-01 10:21:17-03:00       
System Total Memory Is: 867837916 bytes                          
Total Memory Used Is: 203704524 bytes                             
Memory Using Percentage Is: 23%

Generally, high memory usage caused by large service volume or new configurations will remain at a specific value without changing significantly when the service volume is stable. Reducing the service volume can relieve the problem of high memory usage. If memory usage does not continuously increase, run commands in the diagnostic view to confirm related services.
Versions earlier than V200R005
Run the display memory command in the diagnostic view (instead of the display memory-usage command in the user view) to check memory usage.

[HUAWEI] diagnose
[HUAWEI-diagnose] display memory
Block Memory Status
Block Size    32        Free   528      Used   79716    Total   80244
Block Size    64        Free   85       Used   63680    Total   63765
Block Size   128        Free   161      Used  101494    Total  111655
Block Size   256        Free   114      Used    4066    Total    4180
Block Size   512        Free   315      Used     605    Total     920
Block Size  1024        Free    15      Used     645    Total     660
Block Size  2048        Free    44      Used     421    Total     465
Block Size  4096        Free    61      Used     269    Total     330
-----------------------------Summary--------------------------------
Used(Byte)22936832      Free  1323      Used  1170896    Total  1172219
Memory total used size: 22936832        Used Ratio For Memory : 95%

The preceding command output shows that block 128 has occupied the most memory of about 12,991,232 (128*101494). Then run the display inspect mem-debug-info 29 0 0 0 command in the diagnostic view to check the 128-byte memory usage and find out the MID that uses the most memory blocks. The following command output shows that the module 7012 uses the most number of blocks 128.

[HUAWEI] diagnose
[HUAWEI-diagnose] display inspect mem-debug-info 29 0 0 0  
( MID, BLK32, BLK64, BLK128, BLK256, BLK512,BLK1024,BLK2048,BLK4096, DOS, DOS(B))
===============================================================
(7012, 214, 4, c7388, 24, 6, 14, 1, a7, 18, 14b970)

V200R005 and later versions
Some switches use a new memory algorithm. Therefore, the method of collecting memory usage information changes. If a message similar to the following is displayed during the preceding operations,

[HUAWEI-diagnose] display memory
Info: The function is not supported on this board, please try "display dlmalloc-memory".

Or,

[HUAWEI-diagnose] display inspect mem-debug-info 29 0 0 0
Info: The function is not supported on this board, please try "display inspect dlmalloc-mem-info".

Run the display inspect dlmalloc-mem-info 27 1 0 0 0 command in the diagnostic view to check memory usage on each module and find out the module with high memory usage.

[HUAWEI-diagnose] display inspect dlmalloc-mem-info 27 1 0 0  0
=========================================================== 
***********************************************************
HandleID                     HandleName       UsedMemory(B)
-----------------------------------------------------------
0                           vos.sys         21,048,864
24                VOS_PID_RESERVE_24                  0
101                vos.handle.invalid             16,080
104                       VRPS.MID.50            540,520
106                     VRPS.MID.4053            873,676
108                     VRPS.MID.40c1             26,108
110                     VRPS.MID.4048       98,423,080,0
112                        VRPS.MID.5             22,020
114                     VRPS.MID.ffbc             72,596
116                     VRPS.MID.ff0a              1,484
118                     VRPS.MID.411c          4,126,560

The preceding command output shows that the modular 4048 (MID.4048) uses the most memory.
Find out related services based on the MID. One type of services may correspond to multiple MIDs; therefore, you need to find out specific services and then check whether this type of services causes high memory usage. The following describes mappings between MIDs and services:
MIDs d01d, f061, f0b2, and f0b3 correspond to routing. You can reduce routing configurations to check whether memory usage is reduced.
The MID 4048 is often used in user login or by VLANIF interfaces to apply for memory. You can reduce the number of VLANIF interfaces or user logins to check whether memory usage is reduced.
If memory usage continuously increases and cannot be reduced after related configurations are reduced, collect related information for further analysis. Memory usage may periodically increase; therefore, you need to observe it for a long period, for example, collect memory usage information at an interval of one day.
Switch Stacking Diagram

3. Collecting Information and Seeking Technical Support

If the fault persists, collect related information and seek technical support.
Collecting Fault Information
Collect operation results of the preceding steps and record the results in a file.
Collect memory abnormality information.
After finding out the memory block and module that use much memory, you still need to collect memory usage information, which varies depending on system software versions:
In V100R003 of modular switches or version later than V100R005 of fixed switches:
If high memory usage occurs in a fixed-size memory block, run the display inspect mem-debug-info 34 mid-hex hex-string 0 command.
If high memory usage occurs in a dynamic memory block, run the display inspect mem-debug-info 33 mid-hex 0 0 command.
For example:

[HUAWEI-hidecmd] display inspect mem-debug-info 34 128 7012 0
*****************************************************************************
-------------------------------------------------------------------------
index   mid    number   occupy         address            line  filename(Block memory)
(sec)
-------------------------------------------------------------------------
0       7012   815974   649777617       0x0d59f01c        293   rm_tmrpb.c
1       7012    27      736322639       0x0aeeec58        163   rm_dbpb1.c
2       7012    5       736322638       0x0aeec080        548   rm_im.c
3       7012    1       736322639       0x0aef1d38        193   rm_dbpb1.c
4       7012    1       736322627       0x0af10ab8        576   rm_im6.c
*****************************************************************************

In the command output, the content in bold shows that line 293 in the rm_tmrpb.c file uses 815974 memory blocks. Reflect this information to technical support personnel for fault location.
In V200R005 of modular switches or version later than V200R006 of fixed switches:
Run the display inspect dlmalloc-mem-info 5 1 mid shownumbers command to collect data and reflect it to technical support personnel for analysis.
For example:

[HUAWEI-diagnose] display inspect dlmalloc-mem-info 5 1 ff00 10
========================================================== 
********************************************************************************
PtNo  [         1] Start_Addr[0x95ed2b5c]  Ref[         1]
Handle[       121]       Size[       108] Tick[   5004143]
File[vos_adp9300.c:2924]
********************************************************************************
********************************************************************************
PtNo  [         1] Start_Addr[0x95ed2e54]  Ref[         1]
Handle[       121]       Size[       108] Tick[   5004123]
File[vos_adp9300.c:2924]

Collect all diagnostic information and export the information to a file.
Run the display diagnostic-information file-name command in the user view to collect diagnostic information and save the information to a file.

<HUAWEI> display diagnostic-information dia-info.txt
Now saving the diagnostic information to the device
 100%
Info: The diagnostic information was saved to the device successfully.

When the diagnostic file is generated, you can export the file from the device using FTP, SFTP, or SCP.
NOTICE:
You can run the dir command in the user view to check whether the file is generated.
You can also run the display diagnostic-information command and save terminal logs in a diagnostic file on a disk.
If this command displays a long output, press Ctrl+C to abort this command.
This command displays diagnostic information, which helps locate faults but may affect system performance. For example, CPU usage may become high. Therefore, do not use this command when the system is running properly.
Running the display diagnostic-information command simultaneously on multiple terminals connected to the device is prohibited. This is because CPU usage of the device may obviously increase and the device performance may be degraded.

Collect the log and trap information on the device and export the information to files.
Run the save logfile all command in the user view to save the logs in the user log buffer area and diagnostic log buffer area to the user log file and diagnostic log file, respectively.

<HUAWEI> save logfile all
Info: Save logfile successfully.
Info: Save diagnostic logfile successfully.

When the diagnostic file is generated, you can export the file from the device using FTP, SFTP, or SCP.
NOTE:
You can also run the display logbuffer and display trapbuffer commands to view the log and trap information on the device, and save the information in diagnostic files on a disk. Seek technical support.

If any question, please contact csd@telecomate.com to seek technical support.
NOTE:
Technical support personnel will provide instructions for you to submit all the collected information and files, so that they can locate faults.