You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-07 - 20:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot6s.osadl.org (updated Tue Jul 07, 2026 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3110921160,27sleep041ktimersoftd/011:01:200
104532990,5sleep310452-21tail11:18:003
1574999838,51cyclictest0-21swapper/011:25:430
157499974,37cyclictest0-21swapper/009:53:150
1574999711,14cyclictest0-21swapper/009:33:000
1574999521,9cyclictest31347-21ssh08:38:120
157499944,34cyclictest0-21swapper/009:13:160
1574999424,23cyclictest0-21swapper/009:32:060
1574999412,36cyclictest0-21swapper/011:53:120
157499934,24cyclictest0-21swapper/011:37:570
157499934,24cyclictest0-21swapper/011:37:570
1574999325,15cyclictest0-21swapper/011:03:020
1574999325,15cyclictest0-21swapper/011:03:020
1574999321,13cyclictest0-21swapper/009:23:140
1574999311,21cyclictest0-21swapper/008:53:140
157499925,36cyclictest0-21swapper/010:08:420
157499924,34cyclictest0-21swapper/011:32:560
157499924,28cyclictest0-21swapper/011:18:040
157499923,13cyclictest19476-21tune2fs09:08:050
157499913,26cyclictest0-21swapper/010:53:070
157499905,34cyclictest0-21swapper/009:03:150
157499905,25cyclictest0-21swapper/008:42:570
157499904,78cyclictest0-21swapper/011:59:110
157499903,36cyclictest6599-21grep11:12:560
1574999017,9cyclictest11236-21awk10:32:430
157499895,33cyclictest1227-21runrttasks07:21:230
157499894,34cyclictest0-21swapper/008:48:090
157499894,25cyclictest0-21swapper/010:43:120
1574998918,27cyclictest22882-21strings10:48:080
1574998918,13cyclictest0-21swapper/009:19:520
157499885,24cyclictest0-21swapper/011:31:270
157499885,13cyclictest0-21swapper/010:26:270
157499885,13cyclictest0-21swapper/010:26:270
157499884,34cyclictest0-21swapper/009:38:070
157499884,25cyclictest0-21swapper/008:27:590
157499884,13cyclictest5909-21yum-updatesd-he11:12:420
1574998818,12cyclictest0-21swapper/009:48:060
157499875,32cyclictest0-21swapper/010:15:200
157499875,32cyclictest0-21swapper/009:43:200
157499875,27cyclictest0-21swapper/010:22:440
157499875,27cyclictest0-21swapper/010:22:440
157499875,26cyclictest0-21swapper/010:04:040
157499875,24cyclictest0-21swapper/010:40:190
157499874,25cyclictest0-21swapper/008:13:160
157499874,24cyclictest0-21swapper/010:00:480
1574998725,11cyclictest1227-21runrttasks11:48:180
157499865,32cyclictest0-21swapper/008:35:560
157499864,26cyclictest0-21swapper/010:33:020
157499864,24cyclictest0-21swapper/011:46:240
157499864,24cyclictest0-21swapper/011:46:240
157499854,32cyclictest0-21swapper/007:23:110
157499854,27cyclictest0-21swapper/008:07:490
157499844,33cyclictest0-21swapper/008:22:230
157499844,33cyclictest0-21swapper/008:22:230
157499844,25cyclictest0-21swapper/008:58:560
157499844,15cyclictest0-21swapper/006:41:320
157499844,12cyclictest0-21swapper/007:36:420
157499825,32cyclictest0-21swapper/006:33:120
157499825,13cyclictest0-21swapper/007:13:050
157499804,35cyclictest0-21swapper/006:53:090
157499804,35cyclictest0-21swapper/006:53:090
157499804,33cyclictest0-21swapper/008:08:050
157499804,33cyclictest0-21swapper/007:52:490
157499804,33cyclictest0-21swapper/006:58:070
1574998024,13cyclictest0-21swapper/007:53:130
157499795,29cyclictest18742-21aten2.4-expect07:08:040
200982750,3sleep3361ktimersoftd/309:08:153
1578997324,41cyclictest10018-21grep08:53:181
1578997122,28cyclictest5110-21unixbench_multi09:33:211
1578996921,40cyclictest4324-21ps11:08:111
1578996817,29cyclictest23607-21ssh09:13:181
1578996813,30cyclictest0-21swapper/109:58:571
219499670,4rtkit-daemon2193-21rtkit-daemon11:42:373
219499670,4rtkit-daemon2193-21rtkit-daemon11:42:373
1578996718,42cyclictest20996-21needreboot11:33:051
1578996624,34cyclictest1740-21gltestperf11:53:001
1578996624,33cyclictest7052-21cpuspeed_turbos09:37:591
1578996617,15cyclictest10268-21ssh11:17:591
1578996616,42cyclictest0-21swapper/110:43:531
1578996614,46cyclictest0-21swapper/110:17:551
1578996614,46cyclictest0-21swapper/110:17:551
1578996614,45cyclictest0-21swapper/109:48:411
84192653,40sleep30-21swapper/311:14:233
219499650,4rtkit-daemon24018-21rm10:50:092
219499650,4rtkit-daemon2193-21rtkit-daemon09:40:413
1578996518,41cyclictest18759-21aten_r1power_en09:07:571
1578996518,40cyclictest4712-21latency_hist08:47:471
1578996516,43cyclictest0-21swapper/110:38:341
1578996515,44cyclictest0-21swapper/111:46:161
1578996515,44cyclictest0-21swapper/111:46:161
1578996515,42cyclictest0-21swapper/109:17:561
219499640,3rtkit-daemon2193-21rtkit-daemon08:55:282
157899647,49cyclictest0-21swapper/110:51:171
1578996422,35cyclictest32495-21grep11:03:001
1578996422,35cyclictest32495-21grep11:03:001
1578996422,34cyclictest17738-21cpuspeed_turbos08:13:031
1578996421,27cyclictest3372-21cpuspeed_turbos07:43:051
1578996419,37cyclictest0-21swapper/108:12:431
1578996418,29cyclictest10883-21ssh10:31:271
1578996416,41cyclictest0-21swapper/109:43:051
1578996415,26cyclictest12322-21perl10:33:021
1578996414,43cyclictest0-21swapper/108:51:111
1578996413,45cyclictest0-21swapper/110:16:211
1578996413,44cyclictest2807-21ssh09:31:571
1578996318,38cyclictest24753-21missed_timers10:03:081
1578996317,39cyclictest19383-21ssh09:55:561
157899629,22cyclictest0-21swapper/111:26:441
1578996221,34cyclictest21542-21grep07:13:041
1578996219,35cyclictest30897-21ssh09:23:571
1578996216,24cyclictest16206-21cat07:02:521
1578996214,33cyclictest0-21swapper/112:02:091
219499610,4rtkit-daemon2193-21rtkit-daemon10:41:543
1580996129,13cyclictest0-21swapper/309:48:073
157899619,45cyclictest0-21swapper/108:38:581
157899616,48cyclictest0-21swapper/110:23:311
157899616,48cyclictest0-21swapper/110:23:311
1578996121,34cyclictest15070-21latency_hist09:02:461
1578996118,36cyclictest0-21swapper/111:38:291
1578996118,36cyclictest0-21swapper/111:38:291
1578996116,17cyclictest0-21swapper/107:26:411
1578996113,41cyclictest0-21swapper/110:57:531
1578996113,33cyclictest0-21swapper/106:53:051
1578996113,33cyclictest0-21swapper/106:53:051
157999603,50cyclictest0-21swapper/211:32:592
1578996018,35cyclictest0-21swapper/108:20:341
1578996018,35cyclictest0-21swapper/108:20:341
1578996017,37cyclictest2736-21taskset07:41:421
1578996017,37cyclictest2736-21taskset07:41:421
1578996015,39cyclictest31228-21open_files11:48:081
1578996015,28cyclictest27241-21ssh10:54:311
1578996014,20cyclictest0-21swapper/111:30:401
1578996011,14cyclictest0-21swapper/110:08:001
158099596,35cyclictest0-21swapper/308:53:173
158099594,38cyclictest0-21swapper/309:13:113
158099594,30cyclictest0-21swapper/311:07:593
1578995914,29cyclictest29116-21idleruntime07:28:111
1578995914,26cyclictest15936-21ssh09:03:031
278772588,12sleep20-21swapper/208:33:082
216102583,48sleep321614-21ssh10:47:503
1580995827,11cyclictest12064-21ssh08:57:593
1580995824,15cyclictest0-21swapper/310:03:143
157899586,44cyclictest0-21swapper/108:01:451
1578995824,20cyclictest27863-21cut08:33:071
1578995816,21cyclictest0-21swapper/111:12:491
1578995810,40cyclictest0-21swapper/108:32:471
310612573,12sleep20-21swapper/211:01:112
219499570,6rtkit-daemon2193-21rtkit-daemon06:58:252
158099574,35cyclictest14760-21awk11:23:163
1580995739,11cyclictest19452-21gltestperf07:08:113
1580995718,24cyclictest9325-21missed_timers10:28:083
1580995714,13cyclictest0-21swapper/309:22:453
115002574,47sleep30-21swapper/310:32:543
219499560,6rtkit-daemon2193-21rtkit-daemon07:03:252
219499560,5rtkit-daemon2193-21rtkit-daemon06:53:102
219499560,5rtkit-daemon2193-21rtkit-daemon06:53:102
158099565,28cyclictest1227-21runrttasks06:41:323
158099565,28cyclictest0-21swapper/309:03:093
1580995619,21cyclictest28533-21ssh11:43:473
1580995619,21cyclictest28533-21ssh11:43:473
157999565,45cyclictest0-21swapper/211:23:112
157999564,45cyclictest0-21swapper/208:38:202
157999563,37cyclictest0-21swapper/209:30:022
1578995620,29cyclictest1227-21runrttasks08:06:381
1578995618,32cyclictest7607-21crond07:52:491
1578995618,32cyclictest5415-21latency06:38:161
219499550,6rtkit-daemon2193-21rtkit-daemon11:32:523
219499550,5rtkit-daemon2193-21rtkit-daemon07:13:102
219499550,5rtkit-daemon2193-21rtkit-daemon07:08:102
158099555,29cyclictest0-21swapper/309:59:403
158099555,25cyclictest5701-21perl08:48:063
158099554,33cyclictest0-21swapper/311:48:143
158099554,32cyclictest0-21swapper/311:03:163
158099554,32cyclictest0-21swapper/311:03:163
157999554,44cyclictest0-21swapper/211:57:552
1579995534,12cyclictest26326-21gltestperf09:18:052
157999553,43cyclictest0-21swapper/210:08:012
157999553,37cyclictest0-21swapper/208:44:382
1578995523,26cyclictest1227-21runrttasks07:09:101
1578995517,26cyclictest22871-21fschecks_time08:23:051
1578995517,26cyclictest22871-21fschecks_time08:23:051
1578995514,25cyclictest2940-21iostat_ios06:33:151
1578995513,35cyclictest0-21swapper/107:05:241
311192544,8sleep331223-21smartctl10:13:033
219499540,6rtkit-daemon2193-21rtkit-daemon07:23:022
219499540,5rtkit-daemon2193-21rtkit-daemon11:02:532
219499540,5rtkit-daemon2193-21rtkit-daemon11:02:532
219499540,5rtkit-daemon2193-21rtkit-daemon10:56:232
219499540,5rtkit-daemon2193-21rtkit-daemon07:58:083
219499540,5rtkit-daemon2193-21rtkit-daemon06:33:262
158099545,29cyclictest0-21swapper/310:23:173
158099545,29cyclictest0-21swapper/310:23:173
158099545,29cyclictest0-21swapper/309:27:573
158099545,29cyclictest0-21swapper/307:33:163
158099545,28cyclictest1227-21runrttasks09:55:133
158099545,28cyclictest0-21swapper/306:33:163
158099545,26cyclictest0-21swapper/310:47:283
158099545,26cyclictest0-21swapper/309:24:363
158099544,34cyclictest0-21swapper/311:00:153
158099544,30cyclictest0-21swapper/311:27:563
158099544,30cyclictest0-21swapper/310:56:293
158099544,30cyclictest0-21swapper/307:18:173
1580995422,13cyclictest0-21swapper/308:43:173
1578995437,11cyclictest9489-21cstates06:48:051
219499530,5rtkit-daemon2193-21rtkit-daemon07:53:233
219499530,5rtkit-daemon2193-21rtkit-daemon07:28:092
219499530,5rtkit-daemon2193-21rtkit-daemon07:18:242
219499530,5rtkit-daemon2193-21rtkit-daemon06:43:112
1580995323,11cyclictest1227-21runrttasks11:52:513
158099532,46cyclictest0-21swapper/309:45:023
158099532,35cyclictest0-21swapper/308:37:493
1579995338,9cyclictest28345-21head11:43:162
1579995338,9cyclictest28345-21head11:43:162
1579995337,10cyclictest9060-21needreboot07:53:162
157999533,44cyclictest0-21swapper/208:48:022
157899539,38cyclictest0-21swapper/106:43:091
155772533,43sleep215580-21ssh09:49:192
219499520,5rtkit-daemon2193-21rtkit-daemon09:03:502
219499520,5rtkit-daemon2193-21rtkit-daemon06:48:182
219499520,4rtkit-daemon2193-21rtkit-daemon10:34:242
202502522,43sleep20-21swapper/209:08:192
158099525,29cyclictest0-21swapper/311:57:513
158099525,28cyclictest0-21swapper/310:10:113
158099524,29cyclictest0-21swapper/308:08:253
158099524,29cyclictest0-21swapper/307:48:093
158099524,28cyclictest0-21swapper/308:03:173
158099524,28cyclictest0-21swapper/307:03:203
1580995222,12cyclictest0-21swapper/310:18:143
1580995222,12cyclictest0-21swapper/310:18:143
1580995221,13cyclictest0-21swapper/308:23:163
1580995221,13cyclictest0-21swapper/308:23:163
1580995220,13cyclictest0-21swapper/308:32:593
1580995220,11cyclictest3656-21cut09:33:003
157999524,43cyclictest0-21swapper/211:31:142
157999524,41cyclictest0-21swapper/209:53:022
1579995236,10cyclictest4626-21tr06:38:072
1579995231,14cyclictest0-21swapper/211:53:062
1579995230,8cyclictest28914-21ssh09:22:582
157999523,42cyclictest0-21swapper/208:58:192
157999523,21cyclictest0-21swapper/209:42:282
157999522,45cyclictest0-21swapper/211:08:152
1578995210,36cyclictest0-21swapper/107:22:501
139492520,3sleep3361ktimersoftd/306:58:043
51912512,40sleep00-21swapper/007:47:280
302702513,42sleep31124-21snmpd07:32:073
158099515,27cyclictest0-21swapper/307:13:233
158099514,28cyclictest0-21swapper/308:18:243
158099514,28cyclictest0-21swapper/308:18:243
1580995119,13cyclictest0-21swapper/306:53:243
1580995119,13cyclictest0-21swapper/306:53:243
1579995135,10cyclictest19132-21sed10:43:022
1579995133,10cyclictest3973-21tune2fs09:33:042
157999513,43cyclictest0-21swapper/210:13:332
157999513,25cyclictest0-21swapper/210:18:452
157999513,25cyclictest0-21swapper/210:18:452
1578995111,34cyclictest0-21swapper/107:53:411
157499515,28cyclictest0-21swapper/006:48:240
157499514,28cyclictest0-21swapper/006:43:210
324002502,9sleep20-21swapper/211:50:382
194052502,9sleep30-21swapper/308:16:083
158099505,27cyclictest0-21swapper/307:43:073
158099505,27cyclictest0-21swapper/306:43:273
158099504,27cyclictest0-21swapper/306:52:523
157999505,12cyclictest0-21swapper/209:46:212
1579995033,10cyclictest19138-21grep08:13:232
157999503,40cyclictest0-21swapper/210:27:552
157999503,28cyclictest0-21swapper/211:12:542
157499505,28cyclictest0-21swapper/007:58:240
157499504,28cyclictest0-21swapper/007:28:120
176702492,41sleep00-21swapper/007:03:220
171042492,10sleep20-21swapper/210:39:342
158099495,15cyclictest0-21swapper/307:27:503
158099494,26cyclictest0-21swapper/307:42:503
158099494,26cyclictest0-21swapper/307:42:503
1580994915,13cyclictest0-21swapper/308:32:483
157999494,12cyclictest0-21swapper/209:58:022
157499495,36cyclictest0-21swapper/007:38:160
157499495,36cyclictest0-21swapper/007:38:160
1579994835,8cyclictest6600-21grep10:23:172
1579994835,8cyclictest6600-21grep10:23:172
1579994833,9cyclictest24083-21wc11:38:032
1579994833,9cyclictest24083-21wc11:38:032
1578994813,30cyclictest0-21swapper/107:34:171
74222472,9sleep20-21swapper/207:50:282
1579994732,9cyclictest10914-21grep07:58:072
157999473,36cyclictest0-21swapper/209:15:412
157999473,27cyclictest0-21swapper/211:18:132
1579994711,18cyclictest4367-21kworker/2:007:33:012
1578994712,29cyclictest1686-21cat06:32:521
1574994731,11cyclictest0-21swapper/008:23:220
1574994731,11cyclictest0-21swapper/008:23:220
168282462,10sleep20-21swapper/208:08:422
1579994631,10cyclictest614-21head07:38:002
1579994631,10cyclictest614-21head07:38:002
157999463,26cyclictest0-21swapper/210:03:062
157999463,26cyclictest0-21swapper/208:23:062
157999463,26cyclictest0-21swapper/208:23:062
146002462,9sleep20-21swapper/208:05:442
216872452,9sleep20-21swapper/208:19:462
216872452,9sleep20-21swapper/208:19:462
157999453,25cyclictest0-21swapper/207:47:502
1579994523,8cyclictest24565-21munin-node08:28:142
157499444,32cyclictest0-21swapper/006:32:530
15172434,33sleep20-21swapper/206:32:402
1580993816,9cyclictest0-21swapper/306:32:523
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional