You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-27 - 13:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot6s.osadl.org (updated Thu Nov 27, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8230999021,9cyclictest1243-21snmpd21:34:540
823099883,77cyclictest0-21swapper/021:25:500
823099864,75cyclictest0-21swapper/021:09:490
823099863,11cyclictest0-21swapper/023:44:460
8230998618,12cyclictest0-21swapper/021:30:030
823099844,72cyclictest0-21swapper/023:07:440
823099843,29cyclictest0-21swapper/000:02:570
8230998321,9cyclictest9692-21telnet20:04:430
823099824,72cyclictest0-21swapper/020:59:490
8230998224,9cyclictest8818-21timerwakeupswit20:00:070
8230998223,52cyclictest0-21swapper/020:50:000
8230998215,59cyclictest8210-21idleruntime-cro18:54:350
823099814,69cyclictest0-21swapper/023:49:510
823099814,12cyclictest0-21swapper/021:54:560
823099813,26cyclictest0-21swapper/023:16:440
8230998119,8cyclictest29926-21munin-node19:39:550
823099803,26cyclictest0-21swapper/018:54:580
823099803,12cyclictest0-21swapper/021:04:500
823099803,12cyclictest0-21swapper/020:44:420
8230998026,47cyclictest1369-21runrttasks21:14:490
8230998024,49cyclictest0-21swapper/000:22:370
8230998022,12cyclictest7983-21grep23:29:430
8230998015,12cyclictest0-21swapper/022:09:280
8230998015,12cyclictest0-21swapper/022:09:280
823099799,9cyclictest114550irq/46-eth022:35:010
823099798,12cyclictest0-21swapper/020:24:510
823099795,11cyclictest0-21swapper/020:24:310
823099794,67cyclictest0-21swapper/023:35:330
823099794,67cyclictest0-21swapper/023:35:330
823099794,12cyclictest0-21swapper/022:09:400
8230997922,7cyclictest1369-21runrttasks20:56:290
823099784,67cyclictest0-21swapper/019:25:190
823099783,63cyclictest0-21swapper/022:40:270
823099783,26cyclictest10740-21munin-node22:29:480
8230997724,8cyclictest5908-21latency19:54:580
8230997723,11cyclictest0-21swapper/020:30:010
8230997721,12cyclictest0-21swapper/023:19:560
8230997719,12cyclictest0-21swapper/021:48:510
8230997718,10cyclictest1369-21runrttasks21:20:020
823099766,11cyclictest0-21swapper/023:54:440
823099763,11cyclictest0-21swapper/019:29:520
8230997624,9cyclictest1369-21runrttasks20:34:520
8230997624,45cyclictest15607-21munin-node19:10:000
8230997612,9cyclictest7150irq/45-ahci00:19:220
8230997612,9cyclictest7150irq/45-ahci00:19:220
8230997612,10cyclictest1243-21snmpd00:13:280
823099758,9cyclictest114550irq/46-eth022:19:550
823099753,11cyclictest0-21swapper/019:15:050
823099753,11cyclictest0-21swapper/019:15:050
823099753,11cyclictest0-21swapper/019:04:490
8230997519,8cyclictest22536-21munin-run19:24:340
823099744,12cyclictest0-21swapper/022:14:410
823099743,26cyclictest17897-21munin-node22:44:480
823099743,11cyclictest0-21swapper/019:44:470
8230997422,9cyclictest1369-21runrttasks22:03:090
8230997422,8cyclictest1369-21runrttasks23:27:560
8230997422,7cyclictest23120-21cstates20:39:450
8230997422,7cyclictest23120-21cstates20:39:450
8230997421,8cyclictest1369-21runrttasks19:49:510
8230997413,8cyclictest23929-21munin-node21:50:040
8230997412,9cyclictest11897-21munin-node23:39:570
8230997322,8cyclictest19241-21aten_r1power_po21:39:400
8230997310,9cyclictest114550irq/46-eth022:51:240
8230997310,9cyclictest114550irq/46-eth022:51:240
823099728,9cyclictest114550irq/46-eth000:06:460
823099723,13cyclictest0-21swapper/022:54:360
823099719,9cyclictest114550irq/46-eth022:24:460
823099719,17cyclictest0-21swapper/022:59:440
823099714,30cyclictest0-21swapper/020:14:490
823099714,28cyclictest0-21swapper/023:10:000
823099713,16cyclictest0-21swapper/019:04:350
823099693,30cyclictest0-21swapper/019:35:080
823699666,52cyclictest0-21swapper/321:54:473
243799650,5rtkit-daemon2436-21rtkit-daemon19:48:402
8233996438,11cyclictest0-21swapper/121:04:571
823099643,28cyclictest0-21swapper/020:10:340
823699636,50cyclictest0-21swapper/323:12:443
823699634,51cyclictest0-21swapper/319:04:553
823599624,9cyclictest19665-21unixbench_multi20:30:052
823399625,13cyclictest0-21swapper/120:34:431
8233996235,12cyclictest0-21swapper/119:44:591
823699615,49cyclictest0-21swapper/321:34:303
823599614,49cyclictest31634-21grep19:40:082
823699604,49cyclictest0-21swapper/322:24:523
823599604,49cyclictest0-21swapper/223:33:582
823599603,50cyclictest0-21swapper/223:34:492
823599603,50cyclictest0-21swapper/223:34:492
8233996033,12cyclictest25081-21munin-node22:59:491
248802603,11sleep10-21swapper/119:29:071
823699595,47cyclictest10836-21munin-node19:00:063
823699594,49cyclictest0-21swapper/323:54:543
823699594,48cyclictest0-21swapper/322:54:453
823699594,48cyclictest0-21swapper/320:34:323
323212593,49sleep20-21swapper/220:59:412
823699584,47cyclictest22762-21munin-node19:25:023
823699584,47cyclictest0-21swapper/323:39:243
823699584,47cyclictest0-21swapper/323:39:243
823699584,47cyclictest0-21swapper/319:24:353
823699583,48cyclictest7198-21munin-node21:14:503
823599583,48cyclictest0-21swapper/222:34:442
823599583,48cyclictest0-21swapper/220:19:332
823599583,48cyclictest0-21swapper/219:00:042
8233995834,10cyclictest0-21swapper/122:49:511
8233995834,10cyclictest0-21swapper/122:49:511
8233995833,9cyclictest31884-21perl22:04:461
8233995833,9cyclictest31884-21perl22:04:461
8233995828,13cyclictest0-21swapper/119:44:341
8233995827,13cyclictest0-21swapper/123:19:261
264242583,47sleep20-21swapper/219:30:012
138892584,46sleep30-21swapper/320:14:563
823599573,47cyclictest25894-21fschecks_time20:44:502
823599573,47cyclictest0-21swapper/221:14:572
823599573,47cyclictest0-21swapper/219:54:532
8233995738,12cyclictest0-21swapper/100:09:241
8233995732,10cyclictest1044-21munin-node22:09:481
8233995727,13cyclictest0-21swapper/120:49:421
8233995723,14cyclictest0-21swapper/121:10:031
243799570,6rtkit-daemon2436-21rtkit-daemon19:10:042
823699564,45cyclictest0-21swapper/321:25:003
823699563,46cyclictest26246-21munin-node00:09:503
8235995638,11cyclictest23534-21grep22:54:452
823599563,46cyclictest3262-21gltestperf21:04:482
823599562,47cyclictest22751-21aten_r1power_cu20:39:412
823599562,47cyclictest22751-21aten_r1power_cu20:39:412
8233995632,9cyclictest3456-21iostat_ios19:49:551
8233995630,12cyclictest0-21swapper/121:20:241
8233995623,15cyclictest0-21swapper/120:29:571
8233995623,14cyclictest0-21swapper/121:35:471
8233995613,14cyclictest0-21swapper/123:54:351
243799560,5rtkit-daemon2436-21rtkit-daemon20:00:022
8236995519,30cyclictest27230-21crond23:04:253
823599554,8cyclictest12-21rcu_preempt22:04:572
823599554,8cyclictest12-21rcu_preempt22:04:572
823599553,46cyclictest1369-21runrttasks22:44:382
823599553,44cyclictest9589-21munin-node21:19:552
823599553,44cyclictest4809-21munin-node21:09:492
823599552,46cyclictest24261-21switchtime19:25:052
823399556,31cyclictest0-21swapper/100:19:561
823399555,31cyclictest0-21swapper/119:10:051
823399555,30cyclictest0-21swapper/119:04:481
823399554,30cyclictest0-21swapper/121:59:381
8233995530,12cyclictest0-21swapper/121:17:421
8233995530,11cyclictest0-21swapper/100:01:131
8233995529,12cyclictest0-21swapper/122:55:411
8233995523,14cyclictest0-21swapper/123:39:371
8233995521,16cyclictest0-21swapper/120:04:421
243799550,6rtkit-daemon2436-21rtkit-daemon21:39:502
243799550,5rtkit-daemon2436-21rtkit-daemon21:48:582
243799550,5rtkit-daemon2436-21rtkit-daemon21:24:512
243799550,5rtkit-daemon2436-21rtkit-daemon19:49:472
20522552,10sleep30-21swapper/319:49:123
823699544,14cyclictest0-21swapper/323:49:353
823699543,45cyclictest4900-21proc_pri22:14:583
823599548,40cyclictest281ktimersoftd/221:59:522
823599543,44cyclictest0-21swapper/222:09:562
823599543,42cyclictest26246-21munin-node00:09:402
823599542,44cyclictest1243-21snmpd20:11:432
823399546,30cyclictest0-21swapper/123:51:421
823399545,29cyclictest0-21swapper/123:09:261
823399543,25cyclictest0-21swapper/123:29:361
8233995428,12cyclictest0-21swapper/100:10:391
8233995422,13cyclictest0-21swapper/122:17:031
8233995422,13cyclictest0-21swapper/119:14:451
8233995422,13cyclictest0-21swapper/119:14:451
243799540,5rtkit-daemon2436-21rtkit-daemon21:34:432
823699534,10cyclictest17680-21idleruntime-cro22:44:263
823699533,44cyclictest6001-21needreboot21:09:543
823699533,43cyclictest23929-21munin-node21:50:033
8236995321,13cyclictest0-21swapper/321:04:463
8235995337,10cyclictest4316-21grep22:14:482
823599532,10cyclictest0-21swapper/223:19:352
823399536,29cyclictest0-21swapper/123:48:221
823399535,30cyclictest0-21swapper/119:59:551
823399535,29cyclictest0-21swapper/123:34:441
823399535,29cyclictest0-21swapper/123:34:441
823399534,30cyclictest0-21swapper/120:25:031
8233995331,8cyclictest23552-21sshd21:47:471
8233995330,9cyclictest8481-21telnet18:54:461
8233995329,10cyclictest0-21swapper/100:18:211
8233995329,10cyclictest0-21swapper/100:18:211
8233995324,13cyclictest0-21swapper/122:19:491
8233995323,13cyclictest0-21swapper/122:24:481
8233995322,13cyclictest0-21swapper/120:59:511
8233995320,14cyclictest0-21swapper/119:21:351
8233995310,23cyclictest26700-21timerandwakeup20:45:031
70712530,3sleep17143-21cat18:49:551
281752533,11sleep10-21swapper/121:55:431
243799530,5rtkit-daemon2436-21rtkit-daemon20:49:452
243799530,5rtkit-daemon2436-21rtkit-daemon20:24:462
823699525,28cyclictest0-21swapper/300:19:413
823699524,42cyclictest0-21swapper/320:04:463
823699524,30cyclictest0-21swapper/322:44:553
823699523,44cyclictest13424-21threads23:39:553
823699523,42cyclictest1243-21snmpd20:46:483
823699523,42cyclictest1243-21snmpd18:56:313
8235995236,10cyclictest14722-21df23:44:392
823599523,41cyclictest0-21swapper/223:39:432
823399526,28cyclictest0-21swapper/120:21:441
823399526,27cyclictest10893-21aten2.4-expect18:59:461
823399525,29cyclictest0-21swapper/122:37:061
823399525,29cyclictest0-21swapper/121:25:021
823399525,29cyclictest0-21swapper/119:29:511
823399524,38cyclictest0-21swapper/123:24:251
823399524,30cyclictest0-21swapper/123:09:551
823399524,29cyclictest0-21swapper/120:54:531
823399524,29cyclictest0-21swapper/120:39:471
823399524,29cyclictest0-21swapper/120:39:471
823399524,29cyclictest0-21swapper/120:14:401
823399523,28cyclictest0-21swapper/119:59:331
8233995222,13cyclictest0-21swapper/122:40:021
8233995221,13cyclictest0-21swapper/122:47:081
8233995220,14cyclictest0-21swapper/121:50:041
243799520,5rtkit-daemon2436-21rtkit-daemon18:54:492
823699517,38cyclictest0-21swapper/323:59:533
823699516,29cyclictest0-21swapper/319:09:563
823699514,42cyclictest0-21swapper/319:59:333
823699513,41cyclictest6121-21telnet22:19:413
823699513,41cyclictest23858-21munin-node00:04:543
823699513,41cyclictest20874-21df22:49:423
823699513,41cyclictest20874-21df22:49:423
823699513,40cyclictest1243-21snmpd21:01:513
8235995134,11cyclictest20725-21tr20:34:452
8235995133,10cyclictest9546-21needreboot22:24:502
823599512,42cyclictest29690-21latency_hist19:39:342
823399515,28cyclictest0-21swapper/120:09:431
823399515,27cyclictest0-21swapper/122:32:061
243799510,5rtkit-daemon2436-21rtkit-daemon21:29:512
243799510,5rtkit-daemon2436-21rtkit-daemon19:04:562
199562512,9sleep30-21swapper/319:17:123
199562512,9sleep30-21swapper/319:17:123
127662512,10sleep30-21swapper/322:32:533
823699505,40cyclictest0-21swapper/300:19:223
823699505,40cyclictest0-21swapper/300:19:223
823699505,39cyclictest0-21swapper/323:04:413
8236995035,9cyclictest15906-21cpuspeed_turbos20:24:423
823699502,42cyclictest27544-21munin-node20:49:443
8235995035,9cyclictest29881-21head23:09:392
823599503,41cyclictest0-21swapper/200:15:422
823599503,41cyclictest0-21swapper/200:15:422
823599503,39cyclictest4722-21munin-node23:24:422
823399504,27cyclictest0-21swapper/123:24:491
823399503,33cyclictest0-21swapper/121:39:411
256922502,9sleep20-21swapper/200:05:182
149662502,10sleep30-21swapper/320:19:353
119552503,41sleep211997-21nfs_client22:29:542
823699493,39cyclictest84162sleep323:29:513
823699492,43cyclictest0-21swapper/319:40:183
8235994933,9cyclictest17374-21grep23:49:412
8235994933,11cyclictest21902-21cat22:49:582
8235994933,11cyclictest21902-21cat22:49:582
8235994930,11cyclictest16676-21cat22:39:512
823599493,13cyclictest0-21swapper/222:19:402
8233994919,12cyclictest0-21swapper/121:31:481
8233994917,10cyclictest27541-21munin-node19:34:471
42322492,40sleep30-21swapper/323:21:103
318092492,9sleep30-21swapper/320:55:553
187522492,10sleep30-21swapper/321:37:393
823699484,36cyclictest0-21swapper/323:44:563
823699484,24cyclictest28982-21sensors_temp19:35:043
823699483,38cyclictest25247-21telnet19:29:453
823699482,11cyclictest0-21swapper/321:19:453
8235994833,9cyclictest29168-21unixbench_multi23:04:582
823599483,40cyclictest0-21swapper/219:15:032
823599483,40cyclictest0-21swapper/219:15:032
823599482,40cyclictest30034-21telnet20:54:422
823599482,38cyclictest9625-21telnet20:04:422
248452482,40sleep30-21swapper/320:43:543
248452482,40sleep30-21swapper/320:43:543
209072482,39sleep20-21swapper/223:55:142
823699474,37cyclictest0-21swapper/323:14:583
8236994716,13cyclictest0-21swapper/319:50:023
8235994731,10cyclictest22142-21unixbench_singl19:20:082
823599473,38cyclictest21592-21telnet23:59:362
77882473,38sleep30-21swapper/319:59:523
32312473,38sleep30-21swapper/322:13:063
305972473,38sleep30-21swapper/322:01:133
211922472,10sleep30-21swapper/321:43:233
150532472,10sleep30-21swapper/322:36:363
823699464,36cyclictest22404-21tail21:44:503
823699463,38cyclictest0-21swapper/323:29:243
823699463,37cyclictest0-21swapper/320:39:323
823599464,8cyclictest32437-21find00:19:532
823599462,12cyclictest0-21swapper/222:59:502
823599461,12cyclictest0-21swapper/221:49:402
823599461,12cyclictest0-21swapper/220:22:122
7062462,10sleep30-21swapper/322:08:353
7062462,10sleep30-21swapper/322:08:353
281812462,10sleep20-21swapper/221:55:482
8235994530,9cyclictest477-21tune2fs23:14:442
823699442,38cyclictest0-21swapper/320:11:443
823599423,12cyclictest0-21swapper/218:54:362
81052424,6sleep30-21swapper/318:53:223
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional