You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-26 - 17:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot6s.osadl.org (updated Wed Nov 26, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1024521170,2sleep2281ktimersoftd/209:35:122
30037999512,75cyclictest0-21swapper/010:45:180
3003799924,11cyclictest0-21swapper/011:34:520
30037998930,8cyclictest30299-21ps10:15:260
3003799883,32cyclictest18118-21cat07:35:250
30037998733,8cyclictest20246-21grep07:40:210
189772870,10sleep33004199cyclictest11:00:133
30037998618,12cyclictest0-21swapper/008:03:300
30037998614,13cyclictest0-21swapper/007:05:030
3003799853,33cyclictest0-21swapper/009:54:590
30037998523,9cyclictest1243-21snmpd11:35:140
30037998431,12cyclictest32210-21sed10:20:150
30037998421,12cyclictest0-21swapper/010:35:300
3003799833,32cyclictest0-21swapper/010:00:170
3003799833,25cyclictest0-21swapper/010:50:260
30037998324,9cyclictest15836-21aten2.4-expect10:55:050
30037998324,9cyclictest15836-21aten2.4-expect10:55:050
30037998317,11cyclictest0-21swapper/012:05:110
3003799814,33cyclictest0-21swapper/009:50:100
3003799813,27cyclictest0-21swapper/011:20:240
3003799804,35cyclictest0-21swapper/011:45:160
3003799804,32cyclictest0-21swapper/012:23:270
3003799804,25cyclictest0-21swapper/008:40:330
3003799804,25cyclictest0-21swapper/008:40:330
3003799803,36cyclictest4212-21tr08:15:150
3003799803,32cyclictest0-21swapper/009:30:270
3003799803,26cyclictest0-21swapper/006:55:360
30037998021,14cyclictest0-21swapper/011:59:510
3003799794,37cyclictest0-21swapper/007:49:380
3003799793,32cyclictest0-21swapper/011:09:540
30037997921,9cyclictest1311-21munin-node10:25:100
30037997919,8cyclictest12290-21munin-node07:25:210
3003799783,12cyclictest0-21swapper/012:10:140
30037997829,8cyclictest11539-21unixbench_singl11:50:250
3003799773,32cyclictest0-21swapper/007:20:230
30037997723,12cyclictest0-21swapper/009:01:260
3003799764,32cyclictest0-21swapper/009:05:110
3003799764,31cyclictest0-21swapper/010:10:120
3003799764,31cyclictest0-21swapper/010:10:120
3003799763,24cyclictest0-21swapper/009:45:230
30037997422,8cyclictest1243-21snmpd07:34:350
30037997416,13cyclictest0-21swapper/006:55:040
3003799733,9cyclictest5986-21munin-run10:34:540
3003799705,14cyclictest0-21swapper/010:05:190
30037997016,8cyclictest0-21swapper/008:20:130
231172603,12sleep30-21swapper/310:00:253
30039995818,23cyclictest1369-21runrttasks07:00:431
3004099574,14cyclictest0-21swapper/211:30:172
52942563,45sleep10-21swapper/107:10:151
52942563,45sleep10-21swapper/107:10:151
3003999565,12cyclictest985-21munin-node11:30:131
264262563,7sleep326457-21missed_timers11:15:173
243799560,6rtkit-daemon2436-21rtkit-daemon11:55:192
3004099554,12cyclictest0-21swapper/210:35:082
3004099554,12cyclictest0-21swapper/207:15:342
3004099553,23cyclictest7490-21aten2.4-expect11:45:042
30039995522,12cyclictest1243-21snmpd11:20:121
285772558,36sleep10-21swapper/106:50:211
28332554,43sleep30-21swapper/309:20:113
243799550,7rtkit-daemon2436-21rtkit-daemon10:50:062
243799550,6rtkit-daemon2436-21rtkit-daemon10:10:082
243799550,6rtkit-daemon2436-21rtkit-daemon10:10:082
243799550,5rtkit-daemon2436-21rtkit-daemon10:40:222
243799550,5rtkit-daemon2436-21rtkit-daemon09:55:232
237322553,45sleep10-21swapper/107:46:421
30041995435,12cyclictest9091-21unixbench_multi11:45:243
3004099545,29cyclictest0-21swapper/206:55:402
3004099544,12cyclictest0-21swapper/208:10:122
30040995417,10cyclictest0-21swapper/211:35:152
3003999544,27cyclictest0-21swapper/108:58:461
3003999543,44cyclictest25775-21tr09:00:131
30039995421,14cyclictest0-21swapper/109:05:241
30039995413,25cyclictest0-21swapper/108:45:121
243799540,5rtkit-daemon2436-21rtkit-daemon10:16:532
243799540,5rtkit-daemon2436-21rtkit-daemon08:25:273
243799540,5rtkit-daemon2436-21rtkit-daemon08:20:203
30041995322,24cyclictest1369-21runrttasks09:25:183
30041995322,24cyclictest1369-21runrttasks09:25:183
3004099535,28cyclictest0-21swapper/210:55:082
3004099535,28cyclictest0-21swapper/210:55:082
30040995337,9cyclictest1165-21lldpd09:10:332
30040995322,14cyclictest0-21swapper/208:40:122
30040995322,14cyclictest0-21swapper/208:40:122
30040995322,13cyclictest0-21swapper/208:15:192
3003999535,29cyclictest0-21swapper/109:44:571
3003999535,28cyclictest0-21swapper/111:05:091
3003999534,28cyclictest0-21swapper/109:24:581
30039995322,13cyclictest0-21swapper/109:50:121
3003799533,33cyclictest0-21swapper/008:25:320
263122533,43sleep10-21swapper/110:09:501
243799530,5rtkit-daemon2436-21rtkit-daemon11:01:062
243799530,5rtkit-daemon2436-21rtkit-daemon08:30:123
243799530,5rtkit-daemon2436-21rtkit-daemon08:05:203
243799530,5rtkit-daemon2436-21rtkit-daemon07:15:303
243799530,5rtkit-daemon2436-21rtkit-daemon07:00:383
225212530,6sleep322522-21expr12:15:093
174852533,43sleep117625-21df_abs07:35:181
309832523,10sleep10-21swapper/109:10:221
30041995228,9cyclictest1369-21runrttasks09:50:253
3004099525,28cyclictest0-21swapper/211:44:202
3004099525,28cyclictest0-21swapper/211:44:202
3004099524,29cyclictest0-21swapper/212:15:192
3004099524,28cyclictest0-21swapper/208:55:262
30040995235,9cyclictest14123-21grep08:35:172
30040995220,14cyclictest0-21swapper/207:45:272
3003999526,27cyclictest0-21swapper/109:59:581
3003999525,29cyclictest0-21swapper/112:20:231
3003999525,28cyclictest0-21swapper/110:55:171
3003999525,28cyclictest0-21swapper/110:55:171
30039995222,13cyclictest0-21swapper/109:30:291
30039995222,12cyclictest0-21swapper/112:19:511
30039995221,12cyclictest0-21swapper/107:54:381
30039995220,13cyclictest0-21swapper/109:35:451
30037995222,23cyclictest0-21swapper/009:40:190
243799520,5rtkit-daemon2436-21rtkit-daemon12:22:032
243799520,5rtkit-daemon2436-21rtkit-daemon11:50:042
243799520,5rtkit-daemon2436-21rtkit-daemon11:10:132
243799520,5rtkit-daemon2436-21rtkit-daemon08:55:333
243799520,5rtkit-daemon2436-21rtkit-daemon08:00:203
243799520,5rtkit-daemon2436-21rtkit-daemon06:55:233
243799520,4rtkit-daemon2436-21rtkit-daemon08:35:123
166782524,11sleep10-21swapper/107:33:161
83412514,41sleep10-21swapper/108:23:541
30041995126,9cyclictest1243-21snmpd11:05:103
30041995119,13cyclictest0-21swapper/309:40:083
3004099515,29cyclictest0-21swapper/210:01:432
3004099515,27cyclictest0-21swapper/209:09:512
3004099515,27cyclictest0-21swapper/209:00:152
3004099514,30cyclictest0-21swapper/208:20:232
3004099514,29cyclictest0-21swapper/211:20:152
3004099514,28cyclictest0-21swapper/209:25:212
3004099514,28cyclictest0-21swapper/209:25:212
3004099514,28cyclictest0-21swapper/208:50:132
3004099514,28cyclictest0-21swapper/208:25:382
3004099514,28cyclictest0-21swapper/208:00:182
3004099514,28cyclictest0-21swapper/207:30:202
3004099514,28cyclictest0-21swapper/207:25:022
30040995120,13cyclictest0-21swapper/211:25:202
30040995120,13cyclictest0-21swapper/210:30:062
30040995119,14cyclictest0-21swapper/212:10:192
3003999515,29cyclictest0-21swapper/109:25:181
3003999515,29cyclictest0-21swapper/109:25:181
3003999515,28cyclictest0-21swapper/110:50:261
3003999515,28cyclictest0-21swapper/110:35:241
3003999515,27cyclictest0-21swapper/111:15:041
3003999515,26cyclictest30903-21fw_packets11:25:131
3003999514,29cyclictest0-21swapper/111:50:061
3003999514,28cyclictest0-21swapper/110:45:061
3003999514,28cyclictest0-21swapper/110:25:231
3003999514,28cyclictest0-21swapper/110:20:181
3003999514,28cyclictest0-21swapper/107:55:211
3003999514,28cyclictest0-21swapper/107:55:211
30039995122,10cyclictest21559-21taskset09:59:561
30039995121,11cyclictest697-21sed08:05:351
30039995121,11cyclictest0-21swapper/110:15:011
30039995119,12cyclictest0-21swapper/108:35:001
284042513,42sleep10-21swapper/110:11:031
284042513,42sleep10-21swapper/110:11:031
25332513,8sleep12598-21latency08:10:241
243799510,5rtkit-daemon2436-21rtkit-daemon12:00:262
129522513,9sleep20-21swapper/210:47:232
3004199503,40cyclictest5286-21head07:10:133
3004199503,40cyclictest5286-21head07:10:133
30041995020,13cyclictest0-21swapper/311:20:123
3004099505,27cyclictest0-21swapper/209:45:192
3004099505,27cyclictest0-21swapper/207:10:122
3004099505,27cyclictest0-21swapper/207:10:122
3004099504,29cyclictest0-21swapper/208:05:202
3004099504,28cyclictest0-21swapper/211:15:232
3004099504,28cyclictest0-21swapper/207:50:282
3004099504,28cyclictest0-21swapper/207:00:262
3004099504,27cyclictest0-21swapper/209:20:092
3004099504,27cyclictest0-21swapper/209:15:312
3004099504,27cyclictest0-21swapper/207:40:242
30040995018,14cyclictest0-21swapper/211:05:142
3003999505,27cyclictest0-21swapper/108:00:221
3003999505,26cyclictest0-21swapper/111:45:131
3003999505,26cyclictest0-21swapper/107:29:341
3003999505,26cyclictest0-21swapper/107:05:381
3003999504,28cyclictest0-21swapper/112:05:031
3003999504,28cyclictest0-21swapper/109:19:581
3003999504,27cyclictest0-21swapper/111:00:051
3003999504,27cyclictest0-21swapper/108:45:001
3003999504,27cyclictest0-21swapper/108:45:001
30039995020,24cyclictest0-21swapper/111:55:041
30039995019,14cyclictest0-21swapper/108:20:001
3003799503,29cyclictest0-21swapper/011:44:520
3003799503,29cyclictest0-21swapper/011:44:520
30037995023,12cyclictest0-21swapper/007:20:030
249432503,41sleep10-21swapper/111:13:061
243799500,5rtkit-daemon2436-21rtkit-daemon10:25:302
243799500,5rtkit-daemon2436-21rtkit-daemon08:45:263
214482503,41sleep10-21swapper/107:43:161
106452503,11sleep10-21swapper/110:43:401
70442493,10sleep10-21swapper/111:43:491
70442493,10sleep10-21swapper/111:43:491
47012493,10sleep10-21swapper/111:39:221
307792492,9sleep30-21swapper/310:15:493
3004199493,40cyclictest0-21swapper/307:06:033
3004099495,26cyclictest0-21swapper/210:04:592
3004099495,26cyclictest0-21swapper/208:49:472
3004099495,25cyclictest0-21swapper/212:08:392
3004099494,28cyclictest0-21swapper/209:50:092
3004099494,27cyclictest0-21swapper/209:40:162
3004099494,27cyclictest0-21swapper/209:30:202
3004099494,27cyclictest0-21swapper/208:30:132
30040994916,12cyclictest0-21swapper/207:28:492
3003999495,27cyclictest0-21swapper/107:24:331
3003999495,25cyclictest0-21swapper/107:19:321
3003999494,28cyclictest0-21swapper/112:00:091
3003799494,39cyclictest0-21swapper/009:15:300
30037994933,10cyclictest12776-21tail08:30:330
3003799493,41cyclictest0-21swapper/007:52:140
3003799493,38cyclictest3662-21latency_hist08:15:020
224952493,10sleep30-21swapper/308:51:333
165092494,38sleep10-21swapper/109:46:411
47622483,10sleep20-21swapper/207:08:222
47352483,10sleep30-21swapper/311:39:483
33982483,10sleep30-21swapper/310:27:303
30041994827,8cyclictest1243-21snmpd10:50:073
30041994826,9cyclictest14672-21munin-node07:30:173
30041994821,13cyclictest0-21swapper/310:10:263
30041994821,13cyclictest0-21swapper/310:10:263
30041994818,12cyclictest0-21swapper/307:41:143
3004099484,27cyclictest0-21swapper/207:35:362
30040994829,11cyclictest32682-21ps10:20:242
30040994818,15cyclictest0-21swapper/207:55:122
30040994818,15cyclictest0-21swapper/207:55:122
3003999485,26cyclictest0-21swapper/108:27:481
3003999483,25cyclictest20615-21munin-node08:50:141
30039994819,11cyclictest0-21swapper/110:30:041
30039994817,13cyclictest0-21swapper/112:10:201
243799480,5rtkit-daemon2436-21rtkit-daemon08:10:573
236542485,37sleep00-21swapper/012:17:210
3004199473,27cyclictest0-21swapper/307:20:143
30041994726,8cyclictest23361-21df11:10:103
30041994725,9cyclictest1243-21snmpd07:39:363
30041994725,8cyclictest1243-21snmpd11:50:163
30041994718,13cyclictest0-21swapper/308:42:303
30041994718,13cyclictest0-21swapper/308:42:303
3003799474,27cyclictest0-21swapper/011:15:120
3003799473,38cyclictest0-21swapper/007:05:300
263032472,39sleep30-21swapper/307:53:583
259922473,9sleep30-21swapper/310:05:413
213312473,10sleep30-21swapper/309:57:153
87332462,37sleep00-21swapper/010:40:080
57062462,9sleep30-21swapper/310:31:293
30041994630,10cyclictest19652-21cut12:10:053
3004199463,36cyclictest0-21swapper/311:40:223
3004199463,36cyclictest0-21swapper/311:40:223
3004199463,26cyclictest0-21swapper/309:49:283
30041994629,11cyclictest32208-21gltestperf10:20:153
30041994629,10cyclictest8329-21gltestperf09:30:193
23142463,35sleep32320-21crond11:34:523
20522462,10sleep30-21swapper/309:16:323
30041994528,11cyclictest30535-21cut09:10:163
30041994524,8cyclictest12163-21munin-node11:55:183
30041994521,9cyclictest29203-21runrttasks09:05:273
30039994517,12cyclictest0-21swapper/108:35:101
3003799453,25cyclictest0-21swapper/008:05:190
30037994523,17cyclictest0-21swapper/008:39:180
287072452,10sleep30-21swapper/307:59:103
287072452,10sleep30-21swapper/307:59:103
80542443,10sleep30-21swapper/310:35:563
3003799444,34cyclictest0-21swapper/008:50:150
30037994430,10cyclictest0-21swapper/007:55:360
30037994430,10cyclictest0-21swapper/007:55:360
3003799443,35cyclictest0-21swapper/009:25:160
3003799443,35cyclictest0-21swapper/009:25:160
3003799443,25cyclictest0-21swapper/011:25:130
30037994425,7cyclictest25095-21munin-run11:14:530
30037994419,11cyclictest0-21swapper/009:36:440
250202443,35sleep00-21swapper/008:58:070
3004199433,32cyclictest8627-21telnet10:40:063
3004199432,34cyclictest25406-21munin-node09:00:303
3003999434,21cyclictest0-21swapper/106:55:171
3003799433,35cyclictest0-21swapper/007:10:150
3003799433,35cyclictest0-21swapper/007:10:150
3003799433,34cyclictest0-21swapper/009:20:310
3003799433,25cyclictest0-21swapper/008:45:180
30037994322,9cyclictest0-21swapper/012:04:510
243799430,5rtkit-daemon2436-21rtkit-daemon07:28:143
178932432,34sleep30-21swapper/310:59:343
178932432,34sleep30-21swapper/310:59:343
116372432,10sleep30-21swapper/309:35:343
3004199423,33cyclictest1243-21snmpd12:20:213
3004199423,33cyclictest0-21swapper/312:00:183
30041994224,12cyclictest12645-21unixbench-2d10:45:273
30041994220,8cyclictest21828-21munin-node07:45:313
30041994220,8cyclictest16934-21munin-node12:05:073
30041994219,8cyclictest1165-21lldpd06:55:043
30041994218,10cyclictest30108-21munin-node11:25:153
3003799423,32cyclictest0-21swapper/009:10:190
30037994215,12cyclictest0-21swapper/011:00:070
243799420,4rtkit-daemon2436-21rtkit-daemon08:15:353
3004099402,22cyclictest0-21swapper/206:55:042
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional