You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-18 - 06:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot6s.osadl.org (updated Thu Jun 18, 2026 00:43:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27104999541,12cyclictest0-21swapper/122:06:351
27104999338,13cyclictest0-21swapper/119:01:251
2710199933,83cyclictest0-21swapper/021:42:510
2710199924,82cyclictest0-21swapper/021:36:340
2710199914,68cyclictest0-21swapper/021:51:230
2710199913,81cyclictest0-21swapper/023:06:150
2710799903,32cyclictest0-21swapper/321:11:273
2710199903,80cyclictest0-21swapper/023:52:360
2710199903,80cyclictest0-21swapper/022:51:280
27107998938,44cyclictest1636-21df22:46:193
2710199894,78cyclictest0-21swapper/022:27:130
2710199894,76cyclictest0-21swapper/023:21:120
2710199894,76cyclictest0-21swapper/023:21:120
2710199893,76cyclictest0-21swapper/000:19:530
27107998832,49cyclictest0-21swapper/323:26:213
2710199884,77cyclictest1227-21runrttasks21:47:160
2710199884,76cyclictest0-21swapper/000:01:580
2710199884,76cyclictest0-21swapper/000:01:580
2710199884,67cyclictest0-21swapper/022:01:320
2710199884,63cyclictest0-21swapper/023:16:130
2710199883,77cyclictest0-21swapper/022:39:280
2710199883,77cyclictest0-21swapper/022:39:280
2710199883,68cyclictest11556-21aten2.4-expect23:01:140
2710199883,64cyclictest0-21swapper/021:01:280
2710799873,76cyclictest858-21cut19:01:383
2710799873,37cyclictest3666-21cut23:36:213
2710199874,76cyclictest0-21swapper/000:10:050
2710199874,75cyclictest0-21swapper/021:20:150
2710199874,75cyclictest0-21swapper/021:20:150
2710199874,65cyclictest0-21swapper/019:51:290
2710199874,65cyclictest0-21swapper/000:11:110
2710199873,69cyclictest0-21swapper/022:41:210
2710199873,69cyclictest0-21swapper/022:41:210
2710799863,75cyclictest0-21swapper/321:26:253
2710799863,28cyclictest0-21swapper/320:51:093
27107998628,50cyclictest0-21swapper/322:56:333
27107998610,12cyclictest0-21swapper/300:12:103
2710199864,76cyclictest0-21swapper/022:16:050
2710199864,75cyclictest0-21swapper/023:56:160
2710199864,75cyclictest0-21swapper/023:56:160
2710199864,75cyclictest0-21swapper/022:46:290
2710199864,74cyclictest0-21swapper/022:23:040
2710199864,74cyclictest0-21swapper/020:46:360
2710799858,20cyclictest0-21swapper/321:10:033
2710199854,74cyclictest13095-21strings21:26:300
2710199854,74cyclictest0-21swapper/022:34:550
2710199854,74cyclictest0-21swapper/021:12:350
2710199854,74cyclictest0-21swapper/020:51:240
2710199854,67cyclictest0-21swapper/021:56:160
2710199854,67cyclictest0-21swapper/021:56:160
2710199854,63cyclictest0-21swapper/023:36:200
2710199853,76cyclictest0-21swapper/023:26:130
2710199853,75cyclictest19432-21tr23:11:270
2710199853,75cyclictest0-21swapper/020:26:320
2710199853,74cyclictest0-21swapper/020:04:070
2710199853,67cyclictest0-21swapper/021:11:080
2710199853,66cyclictest2504-21latency_hist23:36:020
219499850,5rtkit-daemon2193-21rtkit-daemon22:53:503
2710799843,32cyclictest0-21swapper/323:04:503
2710799843,32cyclictest0-21swapper/319:16:113
2710799843,21cyclictest0-21swapper/323:41:043
2710799843,12cyclictest0-21swapper/321:56:423
2710799843,12cyclictest0-21swapper/321:56:423
2710199844,74cyclictest0-21swapper/019:56:240
2710199844,74cyclictest0-21swapper/019:56:240
2710199844,73cyclictest0-21swapper/020:56:500
2710199844,73cyclictest0-21swapper/019:16:340
2710199844,73cyclictest0-21swapper/019:16:340
2710199844,44cyclictest0-21swapper/022:18:230
2710199843,74cyclictest23456-21crond19:51:100
2710199843,74cyclictest0-21swapper/023:46:330
2710199843,73cyclictest15307-21aten2.4-expect21:31:160
2710199843,65cyclictest7078-21df_abs22:06:190
2710199843,64cyclictest0-21swapper/023:42:060
2710799834,12cyclictest0-21swapper/322:16:243
27107998338,40cyclictest0-21swapper/300:03:093
27107998338,40cyclictest0-21swapper/300:03:093
27107998336,40cyclictest0-21swapper/323:47:293
2710799833,73cyclictest0-21swapper/323:31:213
2710799833,72cyclictest0-21swapper/322:11:353
2710799833,33cyclictest0-21swapper/321:16:233
2710799833,33cyclictest0-21swapper/321:16:233
2710799833,32cyclictest0-21swapper/322:31:253
2710799833,31cyclictest0-21swapper/321:31:333
27107998326,9cyclictest7483-21sed22:06:233
27107998326,11cyclictest0-21swapper/320:26:383
27107998326,10cyclictest15856-21unixbench_multi19:31:453
2710199834,71cyclictest0-21swapper/021:21:380
2710199833,74cyclictest0-21swapper/022:57:520
2710799829,13cyclictest0-21swapper/300:20:033
2710799824,12cyclictest0-21swapper/321:54:073
2710799823,71cyclictest0-21swapper/300:06:323
2710799823,35cyclictest0-21swapper/322:46:033
2710799823,35cyclictest0-21swapper/322:46:033
2710799823,31cyclictest0-21swapper/322:02:363
2710799823,22cyclictest23001-21ssh22:30:233
2710799823,12cyclictest0-21swapper/321:02:163
27107998223,10cyclictest1895-21kernelversion20:11:303
27107998212,62cyclictest0-21swapper/320:56:223
2710199826,60cyclictest0-21swapper/019:36:370
2710199823,64cyclictest0-21swapper/019:13:000
2710799814,12cyclictest0-21swapper/320:53:433
2710799813,12cyclictest0-21swapper/323:21:123
2710799813,12cyclictest0-21swapper/323:21:123
27107998126,12cyclictest0-21swapper/322:26:033
27107998125,13cyclictest0-21swapper/323:59:283
27107998125,13cyclictest0-21swapper/323:59:283
27107998122,9cyclictest19042-21gltestperf23:11:213
2710199814,61cyclictest0-21swapper/019:31:400
2710199813,71cyclictest0-21swapper/019:41:420
2710199813,71cyclictest0-21swapper/019:21:450
2710799804,32cyclictest0-21swapper/322:37:513
2710799804,32cyclictest0-21swapper/322:37:513
2710799804,12cyclictest0-21swapper/321:21:293
2710799803,29cyclictest12776-21kernelversion19:26:333
27107998017,7cyclictest20104-21rm21:36:373
2710199805,66cyclictest0-21swapper/018:51:130
2710199804,64cyclictest0-21swapper/020:41:400
2710199804,61cyclictest0-21swapper/019:06:380
2710799793,31cyclictest0-21swapper/319:06:323
2710799793,22cyclictest0-21swapper/321:48:033
27107997916,12cyclictest0-21swapper/319:56:103
2710199794,68cyclictest0-21swapper/020:11:360
2710799785,13cyclictest0-21swapper/321:41:313
2710799783,30cyclictest0-21swapper/319:41:103
2710799783,15cyclictest0-21swapper/323:11:033
27107997822,9cyclictest31241-21timerandwakeup18:56:433
27107997814,8cyclictest10268-21kworker/3:220:41:183
27107997810,14cyclictest0-21swapper/320:31:233
2710199782,70cyclictest548-21latency_hist20:11:100
2710799773,21cyclictest0-21swapper/323:51:373
27107997727,10cyclictest14137-21netstat20:36:333
27107997727,10cyclictest14137-21netstat20:36:333
2710199774,66cyclictest0-21swapper/019:01:450
2710199773,66cyclictest0-21swapper/018:51:340
2710799766,29cyclictest0-21swapper/320:06:393
2710799763,24cyclictest22109-21df_abs23:16:173
2710199763,66cyclictest877-21dbus-daemon18:56:130
2710199754,53cyclictest0-21swapper/019:28:020
2710199753,66cyclictest0-21swapper/020:21:290
27107997428,8cyclictest27367-21aten_r1power_en18:51:243
27107997427,7cyclictest22149-21gltestperf19:46:283
2710199743,65cyclictest0-21swapper/020:16:350
2710799733,27cyclictest0-21swapper/320:19:303
2710799733,12cyclictest0-21swapper/319:18:013
2710799733,12cyclictest0-21swapper/319:18:013
27107997328,9cyclictest6774-21kernelversion20:21:323
2710199734,45cyclictest0-21swapper/020:36:240
2710199734,45cyclictest0-21swapper/020:36:240
2710699724,63cyclictest0-21swapper/221:30:122
27107997028,8cyclictest27121-21kernelversion19:56:313
27107997028,8cyclictest27121-21kernelversion19:56:313
2710199693,54cyclictest0-21swapper/020:31:210
2710699674,47cyclictest0-21swapper/200:00:162
2710699674,47cyclictest0-21swapper/200:00:162
27104996745,15cyclictest0-21swapper/123:46:351
2710699664,55cyclictest0-21swapper/223:26:212
27106996610,25cyclictest30646-21ssh22:41:172
27106996610,25cyclictest30646-21ssh22:41:172
219499660,21rtkit-daemon2193-21rtkit-daemon21:01:242
219499660,19rtkit-daemon2193-21rtkit-daemon22:19:592
2710699654,53cyclictest10337-21fschecks_time23:46:212
2710699645,52cyclictest0-21swapper/222:11:202
2710699645,52cyclictest0-21swapper/222:03:002
219499640,5rtkit-daemon2193-21rtkit-daemon21:41:382
2710699634,53cyclictest30946-21df_abs21:06:222
27106996322,27cyclictest0-21swapper/200:05:592
27106996322,27cyclictest0-21swapper/200:05:592
27106996318,28cyclictest0-21swapper/222:21:422
2710699626,49cyclictest0-21swapper/223:01:222
2710699625,50cyclictest0-21swapper/200:06:292
2710699625,12cyclictest2536-21sed21:11:272
2710699624,53cyclictest0-21swapper/222:26:282
2710699624,52cyclictest0-21swapper/223:36:262
2710699624,51cyclictest0-21swapper/223:41:292
2710699624,51cyclictest0-21swapper/223:06:292
2710699624,51cyclictest0-21swapper/220:51:362
2710699624,48cyclictest0-21swapper/221:46:212
2710699624,40cyclictest0-21swapper/223:11:192
27104996241,13cyclictest0-21swapper/123:29:071
2710499624,12cyclictest0-21swapper/100:06:331
219499620,6rtkit-daemon2193-21rtkit-daemon20:29:552
219499620,4rtkit-daemon2193-21rtkit-daemon22:00:222
219499620,4rtkit-daemon2193-21rtkit-daemon22:00:222
2710699615,51cyclictest0-21swapper/223:21:232
2710699615,51cyclictest0-21swapper/223:21:232
2710699613,51cyclictest0-21swapper/221:36:372
27106996110,30cyclictest0-21swapper/221:21:242
27104996140,13cyclictest0-21swapper/123:31:291
27104996123,20cyclictest15878-21forks20:41:251
2710699605,49cyclictest0-21swapper/220:12:242
2710699605,49cyclictest0-21swapper/219:36:222
2710699605,48cyclictest6653-21taskset22:06:152
2710699604,51cyclictest0-21swapper/222:46:282
2710699604,50cyclictest0-21swapper/221:54:432
2710699603,50cyclictest23635-21cpuspeed_turbos20:56:212
2710499604,13cyclictest0-21swapper/122:57:581
27104996039,12cyclictest28212-21sed00:11:321
27104996037,15cyclictest0-21swapper/120:11:101
23522604,12sleep20-21swapper/223:35:312
2710699596,46cyclictest1227-21runrttasks21:33:252
2710699595,48cyclictest0-21swapper/222:36:062
2710699595,48cyclictest0-21swapper/222:36:062
2710699595,45cyclictest14431-21swap23:51:322
2710699594,49cyclictest0-21swapper/200:11:212
2710699594,48cyclictest0-21swapper/222:56:182
2710699594,11cyclictest0-21swapper/221:16:382
2710699594,11cyclictest0-21swapper/221:16:382
219499590,5rtkit-daemon2193-21rtkit-daemon20:02:192
27104995840,9cyclictest22851-21sort19:46:411
2710499584,14cyclictest0-21swapper/122:51:251
2710499584,13cyclictest0-21swapper/122:36:171
2710499584,13cyclictest0-21swapper/122:36:171
2710699576,29cyclictest0-21swapper/220:46:222
2710499575,31cyclictest0-21swapper/121:01:261
2710499574,15cyclictest0-21swapper/121:26:071
27104995736,14cyclictest0-21swapper/122:25:051
2710499572,49cyclictest0-21swapper/123:58:211
2710499572,49cyclictest0-21swapper/123:58:211
59552563,46sleep10-21swapper/120:21:231
2710699565,46cyclictest0-21swapper/200:16:302
2710699565,45cyclictest1227-21runrttasks22:31:292
27106995621,14cyclictest28240-21sadc20:01:092
27106995621,14cyclictest28240-21sadc20:01:092
2710499566,42cyclictest0-21swapper/123:41:431
2710499564,45cyclictest0-21swapper/119:59:301
2710499564,45cyclictest0-21swapper/119:59:301
2710499564,12cyclictest31137-21ps00:16:261
2710499564,12cyclictest0-21swapper/122:16:161
27104995637,13cyclictest0-21swapper/123:40:031
27104995636,14cyclictest0-21swapper/122:47:021
27104995636,13cyclictest0-21swapper/122:34:141
27104995626,10cyclictest17627-21ssh21:33:371
219499560,6rtkit-daemon2193-21rtkit-daemon18:51:372
2710699553,48cyclictest0-21swapper/222:54:182
27106995521,13cyclictest0-21swapper/219:21:392
2710499555,14cyclictest0-21swapper/123:02:571
2710499554,35cyclictest0-21swapper/123:18:341
2710499554,13cyclictest0-21swapper/122:13:131
27104995538,11cyclictest0-21swapper/121:43:101
27104995537,11cyclictest20756-21fschecks_count20:51:251
27104995536,11cyclictest27821-21runrttasks21:48:471
27104995522,14cyclictest0-21swapper/119:11:431
219499550,6rtkit-daemon2193-21rtkit-daemon19:51:422
2710699545,29cyclictest32425-21cpuspeed_turbos19:01:272
27106995424,9cyclictest5566-21latency_hist20:21:092
2710499545,30cyclictest0-21swapper/121:51:291
2710499545,28cyclictest0-21swapper/120:26:301
2710499545,26cyclictest14198-21aten2.4-expect19:31:231
2710499544,31cyclictest0-21swapper/121:20:201
2710499544,31cyclictest0-21swapper/121:20:201
2710499544,30cyclictest0-21swapper/120:51:081
27104995438,10cyclictest24948-21timerandwakeup20:56:401
27104995435,12cyclictest0-21swapper/123:21:341
27104995435,12cyclictest0-21swapper/123:21:341
27104995435,12cyclictest0-21swapper/100:03:211
27104995435,12cyclictest0-21swapper/100:03:211
27104995434,13cyclictest0-21swapper/121:58:091
27104995434,13cyclictest0-21swapper/121:58:091
27104995425,12cyclictest0-21swapper/122:41:261
27104995425,12cyclictest0-21swapper/122:41:261
219499540,5rtkit-daemon2193-21rtkit-daemon18:56:442
2710699536,39cyclictest0-21swapper/219:48:252
2710699535,28cyclictest0-21swapper/219:11:272
27106995313,14cyclictest0-21swapper/223:17:062
2710499535,30cyclictest0-21swapper/123:12:311
2710499535,29cyclictest0-21swapper/121:11:231
2710499535,29cyclictest0-21swapper/118:51:411
2710499534,30cyclictest0-21swapper/121:26:221
2710499534,28cyclictest0-21swapper/119:06:441
27104995337,10cyclictest0-21swapper/123:51:281
27104995321,13cyclictest0-21swapper/122:26:211
2710499532,46cyclictest0-21swapper/123:06:481
27104995319,15cyclictest0-21swapper/118:56:391
219499530,5rtkit-daemon2193-21rtkit-daemon20:21:262
219499530,5rtkit-daemon2193-21rtkit-daemon19:16:282
219499530,5rtkit-daemon2193-21rtkit-daemon19:16:282
219499530,4rtkit-daemon2193-21rtkit-daemon19:31:282
2710699522,10cyclictest0-21swapper/220:36:352
2710699522,10cyclictest0-21swapper/220:36:352
27106995219,14cyclictest0-21swapper/219:26:252
2710499525,28cyclictest0-21swapper/121:36:261
2710499525,28cyclictest0-21swapper/121:06:281
2710499524,29cyclictest0-21swapper/119:41:301
2710499524,28cyclictest0-21swapper/120:21:091
2710499523,42cyclictest0-21swapper/122:01:291
219499520,5rtkit-daemon2193-21rtkit-daemon20:33:252
2710699515,28cyclictest0-21swapper/219:43:052
27106995120,14cyclictest0-21swapper/220:06:382
27106995119,13cyclictest0-21swapper/219:07:592
2710499518,26cyclictest0-21swapper/119:21:121
2710499518,26cyclictest0-21swapper/119:21:121
2710499513,29cyclictest0-21swapper/119:26:231
2710699504,27cyclictest0-21swapper/220:41:372
2710499504,29cyclictest0-21swapper/120:01:531
2710499504,28cyclictest0-21swapper/119:36:301
2710499504,27cyclictest0-21swapper/120:34:231
2710499494,11cyclictest1857-21iostat20:11:281
27104994917,13cyclictest0-21swapper/120:36:361
27104994917,13cyclictest0-21swapper/120:36:361
27107994831,10cyclictest20601-21grep19:41:433
27104994716,13cyclictest0-21swapper/119:51:391
27107994429,10cyclictest0-21swapper/319:21:243
27106994423,14cyclictest0-21swapper/218:51:142
113582433,35sleep10-21swapper/119:24:151
27107994220,8cyclictest1124-21snmpd20:03:073
27104994211,21cyclictest0-21swapper/118:51:131
27107993717,7cyclictest27186-21cat18:51:133
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional