You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-27 - 07:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot6s.osadl.org (updated Sat Jun 27, 2026 00:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2194991010,5rtkit-daemon2193-21rtkit-daemon20:45:403
19082991014,35cyclictest1953-21ssh21:13:110
19082999819,72cyclictest0-21swapper/019:28:130
1908299975,29cyclictest15002-21rm22:22:030
1908299975,28cyclictest27644-21ssh21:52:500
19082999722,10cyclictest17235-21ssh20:48:140
1908299955,28cyclictest0-21swapper/021:32:570
1908299954,12cyclictest32024-21timerandwakeup23:33:050
19082999520,13cyclictest12221-21diskstats23:52:480
1908299948,27cyclictest0-21swapper/022:48:010
1908299945,81cyclictest0-21swapper/000:02:530
1908299945,81cyclictest0-21swapper/000:02:530
1908299944,22cyclictest21637-21ssh21:42:580
19082999437,50cyclictest29272-21grep22:42:480
19082999437,50cyclictest29272-21grep22:42:480
1908299943,24cyclictest21161-21grep23:17:520
1908299943,24cyclictest21161-21grep23:17:520
1908299943,22cyclictest24100-21munin-node21:47:560
19082999413,13cyclictest4746-21gltestperf21:17:580
19082999413,13cyclictest4746-21gltestperf21:17:580
19082999411,20cyclictest14579-21ssh23:07:560
26742930,6sleep32675-21munin-node19:13:123
26742930,6sleep32675-21munin-node19:13:123
1908299935,21cyclictest0-21swapper/022:53:050
1908299934,30cyclictest0-21swapper/018:47:570
1908299934,25cyclictest6713-21ssh23:43:060
1908299934,23cyclictest0-21swapper/022:33:290
1908499923,81cyclictest0-21swapper/121:38:231
1908299925,25cyclictest0-21swapper/023:05:420
1908299924,33cyclictest0-21swapper/022:38:010
1908299924,28cyclictest0-21swapper/020:43:060
1908299924,24cyclictest27278-21ssh21:03:070
1908299923,26cyclictest0-21swapper/021:08:130
1908299923,12cyclictest8599-21unixbench_singl22:58:090
19082999219,13cyclictest0-21swapper/021:38:000
19082999215,10cyclictest107550irq/46-eth022:12:530
1908299915,12cyclictest1124-21snmpd22:31:510
1908299914,35cyclictest0-21swapper/019:17:430
1908299914,35cyclictest0-21swapper/019:17:430
1908299914,34cyclictest0-21swapper/020:18:090
1908299914,24cyclictest0-21swapper/021:57:520
1908299914,22cyclictest0-21swapper/023:16:480
19082999126,9cyclictest24491-21gltestperf23:22:520
19082999121,14cyclictest0-21swapper/022:07:370
19082999121,14cyclictest0-21swapper/022:07:370
19082999110,21cyclictest19493-1kworker/0:2H21:26:040
1908499903,79cyclictest0-21swapper/123:49:521
1908299905,28cyclictest0-21swapper/023:57:510
1908299905,28cyclictest0-21swapper/023:57:510
1908299904,29cyclictest0-21swapper/020:58:090
1908299904,24cyclictest0-21swapper/023:50:250
1908299904,12cyclictest20340-21processes20:53:080
1908299903,36cyclictest1689-21grep23:37:480
19082999020,11cyclictest0-21swapper/022:25:100
1908499894,78cyclictest0-21swapper/122:41:561
1908499894,78cyclictest0-21swapper/118:57:591
1908499893,78cyclictest0-21swapper/121:21:111
1908499893,78cyclictest0-21swapper/121:21:111
1908299895,27cyclictest0-21swapper/019:53:110
1908299895,24cyclictest0-21swapper/022:12:130
1908299895,13cyclictest0-21swapper/020:27:510
1908299894,15cyclictest1124-21snmpd19:58:030
1908299894,15cyclictest1124-21snmpd19:58:030
19082998918,10cyclictest0-21swapper/000:11:380
19082998917,12cyclictest0-21swapper/020:03:040
19082998915,12cyclictest0-21swapper/021:28:000
19082998911,24cyclictest30808-1kworker/0:2H19:12:430
1908499884,77cyclictest0-21swapper/121:27:591
1908499884,52cyclictest1788-21yum22:48:101
1908499883,78cyclictest0-21swapper/123:27:561
19082998816,12cyclictest0-21swapper/023:29:240
1908499874,76cyclictest0-21swapper/121:26:161
1908499874,76cyclictest0-21swapper/121:08:151
1908499874,60cyclictest0-21swapper/121:42:581
1908499873,77cyclictest0-21swapper/123:00:321
1908299875,15cyclictest0-21swapper/018:58:020
19082998732,10cyclictest18406-21grep19:47:560
1908499864,74cyclictest0-21swapper/122:54:321
1908499863,76cyclictest0-21swapper/120:53:471
1908299864,33cyclictest0-21swapper/019:20:230
19082998627,13cyclictest0-21swapper/019:33:130
1908499854,9cyclictest15715-21grep23:57:521
1908499854,9cyclictest15715-21grep23:57:521
1908499854,74cyclictest0-21swapper/123:54:141
1908499854,74cyclictest0-21swapper/123:12:211
1908499853,75cyclictest0-21swapper/123:44:441
1908499853,75cyclictest0-21swapper/123:38:581
1908499853,75cyclictest0-21swapper/120:42:541
1908499853,71cyclictest0-21swapper/122:17:361
1908499853,51cyclictest0-21swapper/120:22:411
1908299854,25cyclictest0-21swapper/018:44:250
1908299854,12cyclictest0-21swapper/018:53:000
19082998524,10cyclictest7284-21sort19:23:060
1908499843,74cyclictest0-21swapper/121:13:461
1908499843,74cyclictest0-21swapper/119:32:441
1908299844,29cyclictest0-21swapper/019:43:030
1908499834,9cyclictest24054-21munin-node22:33:101
1908499834,72cyclictest0-21swapper/123:22:471
1908499834,72cyclictest0-21swapper/121:49:181
1908499834,52cyclictest0-21swapper/122:28:321
1908499834,37cyclictest0-21swapper/121:53:081
1908499833,73cyclictest0-21swapper/123:33:021
1908499833,62cyclictest0-21swapper/100:06:341
1908499833,62cyclictest0-21swapper/100:06:341
19084998316,42cyclictest29164-21ssh22:42:481
19084998316,42cyclictest29164-21ssh22:42:481
19082998330,10cyclictest27699-21aten_r1power_fr20:07:530
1908499824,71cyclictest0-21swapper/123:22:071
1908499824,71cyclictest0-21swapper/123:22:071
1908499824,71cyclictest0-21swapper/123:07:201
1908499824,60cyclictest0-21swapper/122:19:591
1908499824,45cyclictest22465-21aten_r1power_en20:57:521
1908499823,8cyclictest3054-21ssh22:03:001
1908499823,8cyclictest3054-21ssh22:03:001
1908499823,72cyclictest0-21swapper/122:12:371
1908499823,72cyclictest0-21swapper/100:11:521
1908499823,71cyclictest0-21swapper/122:02:371
1908299829,19cyclictest3-21ksoftirqd/020:24:010
1908299824,23cyclictest0-21swapper/020:32:500
1908499814,7cyclictest67692chrt20:29:151
1908499813,8cyclictest15427-21munin-node22:22:491
1908499813,71cyclictest0-21swapper/121:03:481
1908499813,70cyclictest0-21swapper/120:49:501
1908499813,56cyclictest0-21swapper/121:36:491
1908299815,27cyclictest1227-21runrttasks20:38:130
1908299815,27cyclictest1227-21runrttasks20:38:130
1908499808,49cyclictest16697-21chrt19:43:041
1908499803,70cyclictest0-21swapper/123:14:451
19082998014,12cyclictest0-21swapper/019:38:030
1908499794,68cyclictest0-21swapper/118:50:541
1908499794,67cyclictest0-21swapper/120:38:061
1908499794,67cyclictest0-21swapper/120:38:061
1908499793,68cyclictest0-21swapper/119:51:201
1908299794,13cyclictest0-21swapper/019:03:040
219499780,6rtkit-daemon11424-21df_abs19:32:581
217432780,4sleep229-21ksoftirqd/200:07:352
1908499784,53cyclictest0-21swapper/119:03:091
1908499773,67cyclictest0-21swapper/119:27:441
1908499773,58cyclictest0-21swapper/119:13:041
1908499773,58cyclictest0-21swapper/119:13:041
1908499764,8cyclictest956-21munin-run19:12:431
1908499753,48cyclictest22604-21turbostat19:57:411
1908499734,8cyclictest19983-21munin-node18:43:011
1908499734,61cyclictest0-21swapper/120:32:531
1908499734,50cyclictest29996-21munin-node20:12:561
1908499732,9cyclictest14709-21sendmail_mailst19:38:111
1908499724,48cyclictest0-21swapper/119:22:441
1908499722,7cyclictest0-21swapper/120:03:001
19084997212,40cyclictest0-21swapper/120:11:341
1908499714,60cyclictest0-21swapper/118:42:451
1908499704,58cyclictest0-21swapper/120:23:121
1908499704,58cyclictest0-21swapper/118:57:131
1908499703,51cyclictest0-21swapper/120:02:421
1908499703,51cyclictest0-21swapper/120:02:421
219499660,5rtkit-daemon2193-21rtkit-daemon00:10:093
219499650,5rtkit-daemon2193-21rtkit-daemon22:14:362
219499630,4rtkit-daemon2193-21rtkit-daemon22:18:442
219499630,4rtkit-daemon2193-21rtkit-daemon21:45:452
219499610,4rtkit-daemon2193-21rtkit-daemon22:38:053
19085996124,20cyclictest0-21swapper/222:28:062
19085996020,25cyclictest15672-21munin-node19:43:132
1908699596,46cyclictest21889-21kworker/3:023:06:133
1908699594,48cyclictest0-21swapper/321:52:543
1908699592,50cyclictest0-21swapper/322:49:483
19085995926,28cyclictest0-21swapper/219:57:552
19085995926,28cyclictest0-21swapper/219:57:552
19085995913,10cyclictest107550irq/46-eth021:52:372
33432582,10sleep20-21swapper/221:16:492
219499570,5rtkit-daemon2193-21rtkit-daemon23:57:322
219499570,5rtkit-daemon2193-21rtkit-daemon23:12:493
19086995734,9cyclictest19778-21gltestperf20:52:583
1908599574,24cyclictest0-21swapper/219:29:032
19085995727,12cyclictest27037-21load21:03:022
19085995725,13cyclictest0-21swapper/200:07:322
19085995725,13cyclictest0-21swapper/200:07:322
19085995713,35cyclictest0-21swapper/222:07:592
253322563,13sleep20-21swapper/223:23:062
219499560,5rtkit-daemon2193-21rtkit-daemon22:53:123
219499560,5rtkit-daemon2193-21rtkit-daemon21:32:533
1908699563,45cyclictest0-21swapper/321:22:383
1908699563,45cyclictest0-21swapper/321:22:383
1908599567,33cyclictest0-21swapper/221:53:502
1908599565,31cyclictest0-21swapper/222:38:012
1908599564,35cyclictest0-21swapper/220:48:552
1908599563,49cyclictest0-21swapper/221:08:382
19085995625,12cyclictest0-21swapper/220:58:012
19085995621,12cyclictest23522-21ntp_offset22:33:032
313852554,45sleep20-21swapper/222:45:132
313852554,45sleep20-21swapper/222:45:132
219499550,5rtkit-daemon2193-21rtkit-daemon23:33:033
219499550,5rtkit-daemon2193-21rtkit-daemon22:36:583
219499550,5rtkit-daemon2193-21rtkit-daemon21:38:002
219499550,5rtkit-daemon2193-21rtkit-daemon20:27:552
1908699554,24cyclictest0-21swapper/322:17:553
1908699553,47cyclictest0-21swapper/323:47:323
1908699553,45cyclictest0-21swapper/321:51:113
1908699553,29cyclictest0-21swapper/321:38:353
1908599558,28cyclictest0-21swapper/223:38:082
1908599555,27cyclictest21079-21ssh23:17:522
1908599555,27cyclictest21079-21ssh23:17:522
19085995536,11cyclictest2291-21grep22:02:502
19085995536,11cyclictest2291-21grep22:02:502
19085995520,29cyclictest13963-21munin-node21:33:062
19085995511,26cyclictest0-21swapper/223:27:542
174252555,44sleep20-21swapper/222:23:372
219499540,6rtkit-daemon2193-21rtkit-daemon20:58:473
219499540,5rtkit-daemon2193-21rtkit-daemon23:52:322
219499540,5rtkit-daemon2193-21rtkit-daemon22:59:273
219499540,5rtkit-daemon2193-21rtkit-daemon22:45:573
219499540,5rtkit-daemon2193-21rtkit-daemon22:45:573
219499540,5rtkit-daemon2193-21rtkit-daemon20:43:102
219499540,5rtkit-daemon2193-21rtkit-daemon20:02:562
1908599548,25cyclictest0-21swapper/200:00:522
1908599548,25cyclictest0-21swapper/200:00:522
1908599545,27cyclictest16873-21munin-node23:13:002
1908599544,31cyclictest0-21swapper/222:47:492
1908599544,30cyclictest0-21swapper/219:57:412
19085995437,11cyclictest19764-21unixbench_multi19:48:152
19085995425,10cyclictest5935-21ssh23:42:542
19085995422,13cyclictest0-21swapper/223:33:012
1908599540,10cyclictest30027-21kworker/1:120:12:522
151302543,11sleep20-21swapper/219:38:472
219499530,6rtkit-daemon2193-21rtkit-daemon23:08:573
219499530,5rtkit-daemon2193-21rtkit-daemon23:28:333
219499530,5rtkit-daemon2193-21rtkit-daemon21:11:163
219499530,5rtkit-daemon2193-21rtkit-daemon20:51:543
219499530,5rtkit-daemon2193-21rtkit-daemon20:37:552
219499530,5rtkit-daemon2193-21rtkit-daemon20:37:552
219499530,5rtkit-daemon2193-21rtkit-daemon20:33:102
1908699533,44cyclictest0-21swapper/323:58:203
1908699533,44cyclictest0-21swapper/323:58:203
1908599536,27cyclictest0-21swapper/222:01:142
1908599535,42cyclictest0-21swapper/220:52:582
1908599534,30cyclictest0-21swapper/223:08:002
1908599534,30cyclictest0-21swapper/219:03:052
19085995335,10cyclictest7334-21df22:57:502
19085995321,14cyclictest0-21swapper/223:04:222
19085995321,13cyclictest0-21swapper/218:47:552
19085995320,12cyclictest7143-21irqstats19:23:052
1908299534,29cyclictest0-21swapper/018:42:450
125792533,9sleep30-21swapper/321:28:363
11502532,10sleep30-21swapper/322:01:163
105302532,10sleep30-21swapper/319:31:143
219499520,5rtkit-daemon2193-21rtkit-daemon23:27:333
219499520,5rtkit-daemon2193-21rtkit-daemon22:29:283
213012522,10sleep30-21swapper/318:45:523
1908599526,26cyclictest1227-21runrttasks21:23:442
1908599525,29cyclictest0-21swapper/218:57:592
1908599525,28cyclictest0-21swapper/220:23:042
1908599524,29cyclictest0-21swapper/222:52:472
1908599524,29cyclictest0-21swapper/219:17:562
1908599524,29cyclictest0-21swapper/218:43:072
1908599524,28cyclictest0-21swapper/218:52:592
19085995221,10cyclictest1124-21snmpd21:28:002
19085995211,33cyclictest10977-21tail19:32:542
1908599515,29cyclictest0-21swapper/219:12:572
1908599515,29cyclictest0-21swapper/219:12:572
1908599515,27cyclictest0-21swapper/221:18:102
1908599515,27cyclictest0-21swapper/221:18:102
1908599515,27cyclictest0-21swapper/220:08:122
1908599514,28cyclictest0-21swapper/220:17:572
1908599514,28cyclictest0-21swapper/219:08:172
74642502,10sleep30-21swapper/322:09:113
219499500,5rtkit-daemon2193-21rtkit-daemon22:23:433
219499500,4rtkit-daemon2193-21rtkit-daemon23:20:563
219499500,4rtkit-daemon2193-21rtkit-daemon23:20:563
1908699509,13cyclictest0-21swapper/322:15:433
43222492,40sleep30-21swapper/322:05:033
43222492,40sleep30-21swapper/322:05:033
283802492,11sleep30-21swapper/318:59:513
19086994923,20cyclictest29647-21gltestperf19:03:033
1908699492,24cyclictest0-21swapper/321:13:213
19086994912,24cyclictest0-21swapper/321:26:393
19082994919,11cyclictest0-21swapper/020:12:540
91652483,38sleep30-21swapper/320:34:103
80552483,39sleep38200-21ssh23:47:343
21402482,39sleep30-21swapper/320:21:533
1908699485,35cyclictest0-21swapper/319:48:023
1908699484,38cyclictest0-21swapper/321:42:543
214502472,10sleep30-21swapper/300:07:213
214502472,10sleep30-21swapper/300:07:213
19086994731,10cyclictest12293-21cut23:52:493
1908699464,36cyclictest0-21swapper/323:37:593
19086994630,10cyclictest16416-21fschecks_time19:43:003
19086994618,12cyclictest0-21swapper/321:03:553
68012452,9sleep30-21swapper/320:29:433
273492452,37sleep30-21swapper/320:07:263
249702452,38sleep30-21swapper/320:02:343
249702452,38sleep30-21swapper/320:02:343
1908699453,36cyclictest0-21swapper/320:13:023
19086994529,10cyclictest21324-21cut19:53:003
19085994518,8cyclictest19060-21latency_hist18:42:452
117242452,9sleep30-21swapper/320:41:273
117242452,9sleep30-21swapper/320:41:273
19086994428,10cyclictest3234-21grep20:22:583
19086994422,8cyclictest6116-21munin-node19:23:013
57002432,34sleep30-21swapper/319:20:303
1908699433,11cyclictest0-21swapper/319:11:533
128982432,35sleep30-21swapper/319:35:463
294592423,32sleep30-21swapper/320:08:513
260882422,34sleep30-21swapper/318:56:043
235932412,32sleep30-21swapper/318:49:393
1908699415,30cyclictest0-21swapper/319:37:593
1908699383,20cyclictest19245-21cat18:42:463
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional