You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-14 - 05:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TI
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot6s.osadl.org (updated Tue Oct 14, 2025 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1531297997518,18cyclictest1580456-21/usr/sbin/munin00:05:191
153129999706,19cyclictest1545712-21/usr/sbin/munin20:30:202
1531299996429,30cyclictest0-21swapper/221:05:292
1531299996359,2cyclictest0-21swapper/220:20:022
1531295995610,36cyclictest0-21swapper/020:40:000
1531297995329,9cyclictest1563486-21/usr/sbin/munin22:25:201
1531295995345,5cyclictest1575943-21sendmail23:39:590
1531299995136,9cyclictest0-21swapper/220:40:002
153129599514,38cyclictest0-21swapper/022:15:300
153129599514,38cyclictest0-21swapper/022:15:290
1531299995042,5cyclictest1586267-21sendmail00:40:002
1531297994833,10cyclictest0-21swapper/120:30:001
1531295994837,6cyclictest1534928-21munin-plugin-st19:30:010
1531295994837,6cyclictest1534928-21munin-plugin-st19:30:010
1531299994738,5cyclictest1563289-21munin-plugin-st22:24:592
1531299994736,6cyclictest1550202-21munin-plugin-st21:04:592
153129599474,13cyclictest12-21ksoftirqd/000:30:000
1531297994629,12cyclictest231ktimers/120:30:121
153129799462,4cyclictest0-21swapper/123:10:011
1531295994633,9cyclictest0-21swapper/023:55:010
153129599461,3cyclictest0-21swapper/021:15:010
1531299994533,8cyclictest0-21swapper/223:20:302
1531299994529,6cyclictest0-21swapper/221:25:302
1531299994529,10cyclictest0-21swapper/219:40:212
1531297994538,4cyclictest1581937-21munin-plugin-st00:14:591
1531299994429,11cyclictest0-21swapper/222:40:302
153129999441,38cyclictest0-21swapper/200:05:192
153129799441,4cyclictest0-21swapper/121:55:011
1531295994434,6cyclictest0-21swapper/021:20:000
1531295994431,9cyclictest1569189-21sendmail22:59:590
1531295994428,10cyclictest0-21swapper/019:40:010
1531295994419,19cyclictest0-21swapper/021:40:210
1531295994416,7cyclictest0-21swapper/020:45:010
1531299994334,5cyclictest0-21swapper/221:55:312
1531299994332,6cyclictest0-21swapper/221:00:002
1531299994332,6cyclictest0-21swapper/221:00:002
1531299994331,8cyclictest1565813-21sendmail22:40:002
1531299994327,9cyclictest0-21swapper/220:15:212
1531297994331,7cyclictest1558936-21grep21:55:301
1531297994328,9cyclictest0-21swapper/121:00:231
153129599431,4cyclictest1580390-21latency_hist00:05:010
153129599431,36cyclictest1564324-21/usr/sbin/munin22:30:190
153129599431,36cyclictest1564324-21/usr/sbin/munin22:30:190
153129999425,21cyclictest14-21rcu_preempt20:15:002
1531299994235,3cyclictest0-21swapper/220:50:272
1531299994232,6cyclictest0-21swapper/223:15:282
1531299994231,8cyclictest0-21swapper/223:30:012
1531299994229,6cyclictest0-21swapper/221:15:262
1531297994229,9cyclictest0-21swapper/120:00:191
1531297994226,6cyclictest0-21swapper/121:45:301
1531297994213,11cyclictest1534286-21/usr/sbin/munin19:25:191
1531297994213,11cyclictest1534286-21/usr/sbin/munin19:25:191
1531295994231,7cyclictest1543519-21grep20:15:300
1531295994229,8cyclictest0-21swapper/019:20:300
153129599422,36cyclictest0-21swapper/021:50:310
153129599421,39cyclictest0-21swapper/023:30:010
1531295994212,27cyclictest1547785-21fschecks_time20:45:160
1531299994132,5cyclictest1538509-21apt-config19:50:012
1531299994132,5cyclictest1538509-21apt-config19:50:012
1531299994119,15cyclictest1583922-21/usr/sbin/munin00:25:202
1531297994133,5cyclictest1572471-21unixbench_multi23:15:281
1531297994133,4cyclictest0-21swapper/123:55:291
1531297994124,13cyclictest231ktimers/119:20:131
1531297994123,13cyclictest231ktimers/121:35:121
153129599417,29cyclictest1553858-21sed21:25:290
1531295994134,5cyclictest20950irq/209-8000000.ethernet19:45:240
1531295994134,5cyclictest20950irq/209-8000000.ethernet19:45:240
1531295994133,6cyclictest20950irq/209-8000000.ethernet23:20:000
153129599413,2cyclictest0-21swapper/022:40:010
153129599413,12cyclictest1580522-21/usr/sbin/munin00:05:190
1531295994130,6cyclictest0-21swapper/019:55:210
153129599412,5cyclictest1568287-21grep22:50:310
153129999404,16cyclictest1545423-21apt-get20:30:022
153129999401,36cyclictest1575033-21unixbench_singl23:30:312
153129999401,2cyclictest0-21swapper/222:55:242
153129799405,10cyclictest0-21swapper/121:10:001
1531297994029,6cyclictest0-21swapper/119:15:191
1531297994022,14cyclictest231ktimers/100:30:131
1531297994020,13cyclictest231ktimers/122:50:141
1531295994033,4cyclictest20750irq/251-8000000.ethernet23:00:180
1531295994032,5cyclictest1576796-21munin-plugin-st23:45:000
1531295994032,4cyclictest0-21swapper/022:05:000
1531295994031,5cyclictest0-21swapper/020:00:300
153129599401,3cyclictest1566500-21cut22:40:280
1531295994011,4cyclictest1544854-21/usr/sbin/munin20:25:200
1531299993933,3cyclictest1540253-21cron19:59:592
1531299993930,5cyclictest168450irq/135-8000000.ethernet-tx219:19:592
153129999392,30cyclictest1568221-21unixbench_multi22:50:312
153129999392,27cyclictest0-21swapper/222:25:242
1531297993932,3cyclictest1549347-21latency_hist21:00:011
1531297993932,3cyclictest1549347-21latency_hist21:00:001
1531297993930,5cyclictest0-21swapper/122:05:281
1531297993929,7cyclictest0-21swapper/123:55:011
1531297993928,6cyclictest1549000-21grep20:50:331
1531295993929,5cyclictest0-21swapper/021:50:020
1531295993926,9cyclictest0-21swapper/000:40:020
153129599392,33cyclictest1549048-21unixbench_singl20:50:330
153129599391,5cyclictest0-21swapper/020:35:010
1531299993830,4cyclictest168450irq/135-8000000.ethernet-tx219:20:262
1531299993810,20cyclictest30-21rcuc/200:10:122
153129799384,15cyclictest1543634-21apt-get20:20:011
1531297993830,5cyclictest1544480-21grep20:20:341
1531297993827,6cyclictest1571351-21kernelversion23:10:251
1531297993822,9cyclictest0-21swapper/122:20:201
1531297993822,12cyclictest231ktimers/120:10:181
1531297993821,12cyclictest231ktimers/119:40:151
1531297993817,17cyclictest231ktimers/119:55:141
1531295993828,5cyclictest0-21swapper/000:20:180
1531295993815,18cyclictest0-21swapper/022:10:010
1531295993813,23cyclictest131ktimers/020:10:270
1531299993726,5cyclictest0-21swapper/221:30:202
153129999372,29cyclictest0-21swapper/223:05:232
153129999371,7cyclictest1531991-21unixbench_multi19:10:302
1531297993724,7cyclictest1583615-21grep00:20:311
1531297993724,7cyclictest0-21swapper/119:50:321
1531297993723,9cyclictest0-21swapper/123:20:211
153129799371,2cyclictest0-21swapper/120:10:011
153129599378,24cyclictest1535661-21grep19:30:310
1531295993732,2cyclictest0-21swapper/019:40:200
1531295993731,3cyclictest1532696-21fschecks_count19:15:210
1531295993725,9cyclictest0-21swapper/021:00:230
1531295993725,6cyclictest0-21swapper/022:10:210
153129599372,29cyclictest1579507-21latency_hist00:00:010
1531295993722,11cyclictest0-21swapper/019:50:180
153129599371,5cyclictest1581768-21mailstats00:10:290
153129599371,3cyclictest0-21swapper/021:20:120
1531299993628,4cyclictest0-21swapper/223:45:092
1531299993627,4cyclictest0-21swapper/221:35:272
1531299993625,6cyclictest0-21swapper/221:40:282
153129999361,29cyclictest0-21swapper/221:50:142
1531297993631,3cyclictest22-21rcuc/100:05:021
1531297993626,6cyclictest1564866-21grep22:30:291
1531297993626,6cyclictest1564866-21grep22:30:291
1531297993625,8cyclictest1552117-21sendmail21:19:591
1531297993625,7cyclictest0-21swapper/121:30:281
1531297993624,7cyclictest0-21swapper/120:40:011
153129799362,2cyclictest0-21swapper/121:30:011
1531297993617,15cyclictest231ktimers/119:30:141
1531295993627,6cyclictest1563230-21grep22:20:320
1531295993622,10cyclictest0-21swapper/019:15:000
153129599361,4cyclictest0-21swapper/022:25:210
153129599361,2cyclictest0-21swapper/020:25:000
1531299993526,6cyclictest1536484-21grep19:35:282
1531299993526,4cyclictest0-21swapper/219:30:222
153129799354,15cyclictest1552419-21/usr/sbin/munin21:20:211
1531297993528,3cyclictest1566615-21grep22:40:291
1531297993526,5cyclictest0-21swapper/123:40:181
1531297993522,9cyclictest0-21swapper/120:45:251
1531295993520,9cyclictest15-21rcuc/000:15:150
1531295993520,9cyclictest15-21rcuc/000:15:140
153129599351,32cyclictest0-21swapper/023:30:160
153129599351,22cyclictest1168746-1kworker/0:2H+kblockd19:10:000
1531299993426,4cyclictest168450irq/135-8000000.ethernet-tx220:45:252
1531299993423,7cyclictest0-21swapper/222:45:212
1531299993420,9cyclictest0-21swapper/223:50:212
1531299993420,6cyclictest311ktimers/222:30:232
1531299993420,6cyclictest311ktimers/222:30:222
153129999341,27cyclictest0-21swapper/220:00:232
1531297993429,3cyclictest22-21rcuc/123:40:021
1531297993425,7cyclictest0-21swapper/123:35:001
1531297993425,4cyclictest0-21swapper/119:10:301
1531297993420,8cyclictest0-21swapper/122:10:291
1531297993411,12cyclictest1568540-21/usr/sbin/munin22:55:241
153129599347,4cyclictest0-21swapper/000:30:120
1531295993428,4cyclictest0-21swapper/021:55:270
1531295993426,6cyclictest131ktimers/023:10:200
1531295993426,5cyclictest0-21swapper/022:50:000
1531295993426,4cyclictest20950irq/209-8000000.ethernet21:30:100
1531295993426,4cyclictest0-21swapper/023:45:190
1531295993425,7cyclictest131ktimers/023:05:290
153129599341,30cyclictest1555197-21irqstats21:35:230
153129999331,3cyclictest1585292-21unixbench_multi00:30:292
153129999331,3cyclictest1576441-21kernelversion23:40:252
1531297993323,6cyclictest0-21swapper/121:40:281
1531297993322,8cyclictest1567418-21grep22:45:291
153129599332,5cyclictest1551058-21latency_hist21:10:010
1531299993224,5cyclictest0-21swapper/223:35:292
1531299993223,5cyclictest0-21swapper/221:50:002
1531299993222,5cyclictest0-21swapper/219:25:202
1531299993222,5cyclictest0-21swapper/219:25:202
1531297993221,8cyclictest0-21swapper/119:40:001
1531297993218,11cyclictest0-21swapper/100:35:301
1531299993124,4cyclictest168450irq/135-8000000.ethernet-tx222:00:132
1531299993122,4cyclictest0-21swapper/219:55:012
153129799311,3cyclictest0-21swapper/123:30:011
153129599313,4cyclictest1572896-21fschecks_count23:20:210
1531295993114,12cyclictest0-21swapper/020:05:170
1531299993023,4cyclictest0-21swapper/200:20:042
1531299993020,8cyclictest0-21swapper/220:10:002
153129999301,24cyclictest1666-21snmpd20:42:512
153129999301,23cyclictest0-21swapper/222:05:162
153129799308,17cyclictest231ktimers/123:00:121
1531297993024,4cyclictest0-21swapper/100:25:301
1531297993021,5cyclictest0-21swapper/119:50:021
1531297993021,5cyclictest0-21swapper/119:50:011
1531297993015,9cyclictest0-21swapper/122:20:021
1531297993015,9cyclictest0-21swapper/122:20:021
1531295993019,6cyclictest0-21swapper/021:00:020
1531295993019,6cyclictest0-21swapper/021:00:020
1531299992916,9cyclictest0-21swapper/223:55:202
1531299992916,9cyclictest0-21swapper/200:15:212
1531299992916,9cyclictest0-21swapper/200:15:212
153129999291,26cyclictest0-21swapper/223:00:292
153129999291,25cyclictest1580148-21unixbench_multi00:00:292
1531297992910,14cyclictest231ktimers/122:00:131
153129999282,5cyclictest30-21rcuc/222:15:012
1531299992823,3cyclictest168450irq/135-8000000.ethernet-tx223:10:162
1531299992821,4cyclictest168450irq/135-8000000.ethernet-tx222:15:172
1531299992821,4cyclictest168450irq/135-8000000.ethernet-tx222:15:162
153129799281,25cyclictest0-21swapper/122:35:161
1531297992810,14cyclictest231ktimers/123:45:091
153129799271,4cyclictest0-21swapper/100:15:271
153129799271,4cyclictest0-21swapper/100:15:261
153129999251,13cyclictest0-21swapper/221:22:012
153129799251,22cyclictest0-21swapper/121:11:301
1531299992416,5cyclictest1666-21snmpd21:10:332
153129799241,21cyclictest0-21swapper/120:44:021
1531297992215,3cyclictest1531380-21latency_hist19:10:001
1531299991810,6cyclictest311ktimers/219:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional