You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-23 - 21:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot6s.osadl.org (updated Mon Mar 23, 2026 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2485991570,8rtkit-daemon0-21swapper/209:36:003
2835421200,6sleep228353-21iostat_ios08:01:012
2485991060,8rtkit-daemon0-21swapper/205:41:092
17095991033,13cyclictest1419-21runrttasks07:38:581
170939910226,10cyclictest29081-21sed08:01:120
248599990,4rtkit-daemon0-21swapper/205:36:092
17097999934,48cyclictest98301diskmemload08:56:003
17097999522,12cyclictest0-21swapper/310:35:443
17097999414,13cyclictest0-21swapper/307:51:023
1709399934,44cyclictest19023-21ssh08:35:530
177122920,5sleep317703-21ssh09:21:003
17097999229,8cyclictest8635-21tune2fs09:55:523
1709599924,12cyclictest0-21swapper/111:00:461
1709799893,23cyclictest0-21swapper/309:11:023
1709799884,13cyclictest0-21swapper/311:01:033
17097998824,10cyclictest30417-21cpuspeed_turbos09:40:493
17097998810,21cyclictest0-21swapper/309:46:043
1709799874,22cyclictest0-21swapper/307:40:553
1709799873,23cyclictest0-21swapper/309:34:433
17097998729,12cyclictest0-21swapper/307:59:063
17097998728,7cyclictest9830-21diskmemload08:04:173
17097998716,9cyclictest9830-21diskmemload08:36:543
1709799864,31cyclictest0-21swapper/308:10:413
1709799864,23cyclictest0-21swapper/309:06:293
1709799864,23cyclictest0-21swapper/309:06:293
17097998630,7cyclictest22504-21rm09:28:343
17097998620,9cyclictest17046-21ssh10:55:563
1709799858,21cyclictest0-21swapper/309:50:413
1709799858,21cyclictest0-21swapper/309:50:413
1709799853,24cyclictest0-21swapper/309:02:273
17097998529,11cyclictest0-21swapper/308:30:313
17097998526,12cyclictest0-21swapper/308:34:333
17097998526,12cyclictest0-21swapper/308:34:333
17097998514,9cyclictest0-21swapper/310:19:293
1709799845,24cyclictest0-21swapper/308:47:253
1709799844,30cyclictest0-21swapper/310:05:573
1709799844,26cyclictest0-21swapper/310:35:153
1709799844,26cyclictest0-21swapper/310:35:153
1709799844,12cyclictest0-21swapper/310:52:213
17097998424,9cyclictest1965-21cpuspeed_turbos06:10:583
17097998422,10cyclictest29478-21switchtime07:06:123
17097998415,11cyclictest0-21swapper/310:21:023
1709799834,71cyclictest0-21swapper/308:43:023
1709799834,24cyclictest0-21swapper/309:15:503
1709799834,22cyclictest0-21swapper/310:41:153
1709799833,22cyclictest0-21swapper/308:21:593
17097998329,11cyclictest0-21swapper/307:46:003
17097998327,10cyclictest0-21swapper/310:04:053
17097998324,10cyclictest30695-21aten_r1power_cu08:05:503
17097998318,12cyclictest0-21swapper/310:15:223
1709799824,33cyclictest0-21swapper/310:46:003
1709799824,24cyclictest0-21swapper/307:37:003
1709799823,28cyclictest0-21swapper/307:11:043
1709799823,28cyclictest0-21swapper/307:11:043
1709799823,27cyclictest0-21swapper/308:51:153
1709799823,27cyclictest0-21swapper/308:19:283
1709799823,25cyclictest0-21swapper/306:50:533
17097998216,12cyclictest0-21swapper/307:31:153
1709799814,13cyclictest0-21swapper/305:35:563
17097998130,6cyclictest31527-21cat06:05:453
1709799813,31cyclictest0-21swapper/310:26:493
59132800,4sleep03-21ksoftirqd/009:04:290
1709799794,32cyclictest0-21swapper/305:51:063
17097997923,8cyclictest28021-21missed_timers05:56:103
1709799783,29cyclictest0-21swapper/307:30:423
17097997822,9cyclictest12080-21fschecks_time06:31:033
1709799774,35cyclictest0-21swapper/306:36:013
143452770,2sleep20-21swapper/208:28:272
1709799764,33cyclictest0-21swapper/306:21:053
1709799763,15cyclictest0-21swapper/305:45:553
1709799763,15cyclictest0-21swapper/305:45:553
17097997623,9cyclictest32234-21sshd06:06:013
1709799753,27cyclictest4575-21grep07:21:153
1709799733,22cyclictest0-21swapper/306:26:053
1709799724,22cyclictest0-21swapper/306:41:113
17096997237,13cyclictest0-21swapper/209:24:342
1709799714,31cyclictest0-21swapper/306:55:583
1709399704,58cyclictest0-21swapper/009:45:550
17097996924,9cyclictest0-21swapper/305:42:513
1709599644,10cyclictest16031-21perl08:30:591
1709599644,10cyclictest16031-21perl08:30:591
17097996322,9cyclictest0-21swapper/307:20:093
17095996335,14cyclictest0-21swapper/108:19:471
1709699624,50cyclictest0-21swapper/210:01:552
1709399626,12cyclictest0-21swapper/010:19:210
17096996131,23cyclictest0-21swapper/210:36:012
1709399613,12cyclictest0-21swapper/008:26:050
17096996010,13cyclictest0-21swapper/210:46:412
1709399605,12cyclictest0-21swapper/008:55:580
309802593,11sleep20-21swapper/207:11:002
309802593,11sleep20-21swapper/207:11:002
1709699594,47cyclictest0-21swapper/210:15:562
1709699594,47cyclictest0-21swapper/208:09:302
1709399594,48cyclictest24518-21diskstats07:55:550
1709699589,8cyclictest7728-21tail07:30:532
17096995843,10cyclictest12150-21grep06:31:032
17096995840,10cyclictest28440-21tr10:25:482
1709399584,47cyclictest0-21swapper/010:35:330
1709399584,47cyclictest0-21swapper/010:35:330
1709399584,12cyclictest0-21swapper/007:15:560
161972586,8sleep00-21swapper/009:20:140
267772570,6chrt0-21swapper/205:55:452
17096995740,10cyclictest27393-21head07:01:162
1709699574,45cyclictest0-21swapper/209:40:362
1709699574,24cyclictest0-21swapper/208:10:542
17096995739,11cyclictest7743-21tail09:06:022
17096995739,11cyclictest7743-21tail09:06:022
17096995739,11cyclictest25174-21unixbench_singl09:31:102
1709599573,11cyclictest0-21swapper/108:45:561
1709399574,12cyclictest0-21swapper/008:20:500
17093995713,37cyclictest13439-21sh07:40:480
13322573,47sleep20-21swapper/210:32:402
13322573,47sleep20-21swapper/210:32:402
224272563,10sleep20-21swapper/209:28:182
1709699564,45cyclictest0-21swapper/208:20:482
1709699564,44cyclictest0-21swapper/208:35:582
1709399565,12cyclictest0-21swapper/011:01:040
17093995638,11cyclictest0-21swapper/009:30:570
1709399563,8cyclictest121850irq/46-eth009:06:590
1709399563,8cyclictest121850irq/46-eth009:06:590
1709399563,47cyclictest0-21swapper/008:45:560
138362566,10sleep00-21swapper/010:03:480
123252565,10sleep00-21swapper/007:37:460
221072553,9sleep10-21swapper/109:27:081
1709699559,29cyclictest0-21swapper/210:25:192
1709699555,14cyclictest0-21swapper/207:50:582
1709699554,45cyclictest0-21swapper/209:49:272
1709699554,43cyclictest0-21swapper/209:56:052
17096995534,14cyclictest0-21swapper/209:55:242
17096995534,14cyclictest0-21swapper/209:55:242
1709699553,46cyclictest0-21swapper/208:41:042
1709699553,43cyclictest0-21swapper/210:50:542
17096995513,34cyclictest0-21swapper/207:56:112
17096995512,26cyclictest20383-21interrupts11:00:552
1709599554,11cyclictest0-21swapper/106:05:451
17095995531,19cyclictest0-21swapper/110:09:321
1709599553,47cyclictest0-21swapper/110:45:541
17095995510,11cyclictest0-21swapper/106:56:091
1709399555,12cyclictest0-21swapper/010:46:010
1709399554,12cyclictest0-21swapper/009:20:500
1709399554,12cyclictest0-21swapper/008:40:510
1709399554,10cyclictest16295-21iostat_ios08:31:030
1709399554,10cyclictest16295-21iostat_ios08:31:030
17093995537,12cyclictest0-21swapper/009:50:480
17093995537,12cyclictest0-21swapper/009:50:480
1709399553,9cyclictest13746-21basename06:35:530
1709399553,46cyclictest12612-21munin-node10:50:470
1709399553,46cyclictest0-21swapper/007:46:110
1709399553,45cyclictest32623-21ssh09:43:390
1709399553,45cyclictest1287-21snmpd07:24:540
1709399553,44cyclictest8891-21munin-node06:26:200
41052545,10sleep00-21swapper/006:15:480
264052543,44sleep00-21swapper/005:52:580
248599540,5rtkit-daemon2484-21rtkit-daemon06:31:140
226652543,45sleep022705-21tune2fs05:46:010
226652543,45sleep022705-21tune2fs05:46:010
1709699544,41cyclictest0-21swapper/209:41:072
17096995435,10cyclictest30161-21crond07:10:412
17096995433,12cyclictest0-21swapper/209:10:482
1709699543,26cyclictest13263-21latency_hist07:40:402
17096995423,13cyclictest1419-21runrttasks09:01:382
17096995420,15cyclictest0-21swapper/207:50:192
1709599544,12cyclictest0-21swapper/110:35:501
1709599543,36cyclictest0-21swapper/107:19:241
1709599543,25cyclictest0-21swapper/109:54:251
1709599543,25cyclictest0-21swapper/109:54:251
1709599543,12cyclictest0-21swapper/109:00:591
1709599543,11cyclictest0-21swapper/110:24:471
1709599543,10cyclictest0-21swapper/109:37:581
1709599542,46cyclictest9830-21diskmemload10:02:561
1709399544,43cyclictest6595-21telnet06:20:550
1709399544,16cyclictest0-21swapper/010:37:560
1709399544,13cyclictest0-21swapper/008:09:400
1709399543,45cyclictest9830-21diskmemload09:56:450
1709399543,44cyclictest0-21swapper/007:25:570
1709399542,46cyclictest0-21swapper/006:51:090
95042532,44sleep00-21swapper/007:31:370
1709699535,29cyclictest0-21swapper/207:21:052
1709699535,29cyclictest0-21swapper/205:50:452
1709699535,29cyclictest0-21swapper/205:50:452
1709699535,16cyclictest0-21swapper/208:19:342
1709699534,41cyclictest0-21swapper/208:46:072
1709699534,30cyclictest0-21swapper/210:05:522
17096995335,10cyclictest31398-21sh06:05:442
17096995326,20cyclictest0-21swapper/210:40:362
17096995323,10cyclictest9830-21diskmemload10:10:522
17096995320,14cyclictest0-21swapper/208:50:552
1709599534,10cyclictest0-21swapper/106:26:201
17095995334,12cyclictest0-21swapper/108:00:401
1709599533,44cyclictest9830-21diskmemload08:24:571
1709599533,43cyclictest22549-21ssh08:40:541
1709599533,11cyclictest11542-21timerwakeupswit09:11:071
1709599532,45cyclictest22085-21munin-node05:46:141
1709599532,45cyclictest22085-21munin-node05:46:141
1709399534,41cyclictest19943-21ssh09:25:450
1709399534,15cyclictest0-21swapper/008:18:230
1709399534,13cyclictest0-21swapper/009:12:370
1709399533,45cyclictest0-21swapper/010:27:490
1709399533,44cyclictest0-21swapper/010:42:210
1709399533,44cyclictest0-21swapper/010:12:470
1709399533,43cyclictest27078-21ssh10:22:560
1709399533,13cyclictest0-21swapper/010:09:300
1709399533,13cyclictest0-21swapper/006:11:030
1709399532,9cyclictest121850irq/46-eth009:37:260
154492533,43sleep20-21swapper/209:17:302
248599520,5rtkit-daemon2484-21rtkit-daemon05:56:160
213012524,9sleep10-21swapper/108:39:151
1709699525,29cyclictest0-21swapper/208:55:582
1709699525,29cyclictest0-21swapper/206:35:542
1709699524,29cyclictest0-21swapper/206:56:082
17096995232,12cyclictest675-21grep07:15:562
1709699523,29cyclictest3878-21crond06:15:432
17096995220,11cyclictest16586-21forks10:55:492
1709599523,43cyclictest22419-21rm07:51:261
1709599523,43cyclictest0-21swapper/107:50:411
1709599523,11cyclictest24087-21yum-updatesd-he10:19:231
1709599523,11cyclictest0-21swapper/109:44:571
1709599523,11cyclictest0-21swapper/108:59:051
1709599523,10cyclictest0-21swapper/108:05:581
1709599522,44cyclictest3946-21rm08:12:261
1709599522,41cyclictest1287-21snmpd10:30:241
1709599522,10cyclictest7836-21open_files09:06:041
1709599522,10cyclictest7836-21open_files09:06:041
1709399525,40cyclictest2286-21ssh08:10:550
1709399525,12cyclictest0-21swapper/007:50:570
1709399524,42cyclictest0-21swapper/008:51:090
1709399524,29cyclictest0-21swapper/006:41:080
105222523,10sleep10-21swapper/109:58:571
1709699514,29cyclictest0-21swapper/207:40:572
1709699514,29cyclictest0-21swapper/206:46:162
1709699514,29cyclictest0-21swapper/206:41:122
1709699514,29cyclictest0-21swapper/206:16:022
1709699514,28cyclictest0-21swapper/206:20:572
1709699514,27cyclictest0-21swapper/206:26:042
1709699514,27cyclictest0-21swapper/206:09:422
1709699513,29cyclictest0-21swapper/205:56:132
17096995119,13cyclictest0-21swapper/208:30:552
17096995119,13cyclictest0-21swapper/208:30:552
17095995131,13cyclictest0-21swapper/106:41:041
1709599513,42cyclictest5933-21telnet10:40:451
1709599513,41cyclictest9830-21diskmemload10:53:051
1709599513,41cyclictest1949-21hddtemp_smartct09:45:551
1709599513,41cyclictest13751-21df_abs09:15:521
1709599513,40cyclictest9830-21diskmemload08:25:461
1709599513,40cyclictest25731-21ssh09:33:051
1709599513,11cyclictest0-21swapper/106:51:131
1709599513,10cyclictest0-21swapper/107:41:121
1709399514,29cyclictest0-21swapper/010:59:230
206092503,40sleep00-21swapper/006:50:420
1709699505,28cyclictest21936-21latency06:51:052
1709699504,23cyclictest0-21swapper/207:25:532
17095995035,9cyclictest8527-21gltestperf07:31:011
1709599503,9cyclictest0-21swapper/106:35:531
1709599503,42cyclictest0-21swapper/110:34:521
1709599503,42cyclictest0-21swapper/110:34:521
1709599503,41cyclictest0-21swapper/110:55:441
1709599503,40cyclictest29812-21ssh08:03:301
1709599502,41cyclictest9830-21diskmemload08:50:431
1709399506,26cyclictest0-21swapper/005:41:140
1709399504,28cyclictest0-21swapper/006:00:580
1709399503,43cyclictest0-21swapper/006:59:510
1709399503,42cyclictest0-21swapper/005:40:440
317872492,40sleep10-21swapper/106:05:571
203152492,41sleep10-21swapper/106:46:521
1709599493,11cyclictest0-21swapper/107:14:521
1709599493,11cyclictest0-21swapper/107:14:521
17095994916,15cyclictest0-21swapper/105:38:431
1709399493,42cyclictest0-21swapper/007:01:100
323542483,10sleep00-21swapper/007:13:150
323542483,10sleep00-21swapper/007:13:150
1709799484,24cyclictest5772-21sed06:16:173
1709799483,26cyclictest0-21swapper/307:02:163
1709599484,38cyclictest0-21swapper/109:25:361
17095994833,10cyclictest6141-21sed07:26:011
1709599483,8cyclictest121850irq/46-eth010:12:011
1709599483,41cyclictest0-21swapper/105:51:381
17095994816,14cyclictest0-21swapper/106:14:431
17097994727,9cyclictest0-21swapper/306:47:403
1709599473,36cyclictest29021-21munin-run06:00:441
17095994719,11cyclictest28486-21runrttasks07:05:571
1709399473,40cyclictest0-21swapper/007:06:080
1709399473,37cyclictest0-21swapper/006:10:440
1709599463,36cyclictest11292-21munin-node06:31:111
1709699454,12cyclictest0-21swapper/205:35:452
1709599453,38cyclictest0-21swapper/107:05:421
1709599453,35cyclictest4114-21munin-node06:16:131
1709599453,10cyclictest0-21swapper/107:20:551
1709599442,25cyclictest0-21swapper/105:45:311
1709799433,13cyclictest0-21swapper/305:35:453
1709599433,33cyclictest0-21swapper/106:25:441
1709599413,32cyclictest0-21swapper/105:35:451
1709399363,28cyclictest17069-21latency_hist05:35:450
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional