You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-24 - 16:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Fri Apr 24, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
168482239162,24sleep30-21swapper/307:09:033
168102235162,60sleep00-21swapper/007:08:300
168812234163,58sleep20-21swapper/207:09:262
166242199164,23sleep10-21swapper/107:06:101
2791321540,4sleep11701599cyclictest07:30:221
35992930,3sleep11701599cyclictest11:30:171
17019997571,3cyclictest0-21swapper/212:16:282
17019997471,2cyclictest0-21swapper/210:39:202
255742590,2sleep30-21swapper/308:40:153
293902510,3sleep229394-21unixbench-2d10:00:252
207852440,2sleep30-21swapper/310:55:193
66902420,1sleep30-21swapper/307:55:253
17019994238,3cyclictest0-21swapper/211:28:422
17019994238,3cyclictest0-21swapper/211:28:422
17010994011,14cyclictest31130-21crond12:35:010
1701099389,13cyclictest947-21idleruntime-cro12:40:010
1701099389,13cyclictest947-21idleruntime-cro12:40:010
1701099358,13cyclictest20161-21munin-run12:10:010
1701999341,15cyclictest0-21swapper/209:40:172
1701099339,11cyclictest24895-21perf08:40:010
161552330,5sleep07-21ksoftirqd/008:20:020
291462320,4sleep029148-21gltestperf11:15:140
1701099318,11cyclictest19198-21basename07:15:020
1701099316,12cyclictest11360-21munin-run11:50:010
1701599299,12cyclictest0-21swapper/109:20:141
1702599280,5cyclictest0-21swapper/311:47:453
17019992716,3cyclictest0-21swapper/212:20:402
1701999271,10cyclictest27591-21sendmail_mailqu11:10:232
1701999268,3cyclictest16648-21df08:20:122
17019992614,2cyclictest30984-21taskset12:33:082
1701999261,15cyclictest617-21if_enp2s010:10:162
1701999261,12cyclictest0-21swapper/207:50:132
1701099264,11cyclictest0-21swapper/008:30:020
303762250,2sleep081ktimersoftd/008:50:160
1701599251,5cyclictest5648-40dnf09:09:251
1701599251,17cyclictest0-21swapper/110:24:121
1701599251,16cyclictest15366-21users11:55:261
1702599242,13cyclictest0-21swapper/309:35:253
1702599241,9cyclictest0-21swapper/309:55:083
1701999245,2cyclictest0-21swapper/212:25:242
1701999241,14cyclictest0-21swapper/211:05:242
1701999241,14cyclictest0-21swapper/209:45:252
1701599248,3cyclictest30592-21grep11:20:021
1701599243,3cyclictest7438-21latency10:25:161
1701599241,4cyclictest19187-21hddtemp_smartct08:25:151
1701599240,19cyclictest0-21swapper/112:13:501
1701099242,9cyclictest26434-21kworker/0:212:21:430
1701099241,11cyclictest13348-21kworker/0:111:54:180
1701099240,11cyclictest11711-21idleruntime-cro08:10:020
3042233,2sleep3425-21processes08:55:213
1702599231,5cyclictest0-21swapper/309:30:583
1702599231,4cyclictest0-21swapper/311:16:213
17019992312,3cyclictest0-21swapper/210:45:362
1701999231,14cyclictest0-21swapper/208:05:242
17019992310,7cyclictest21851-21unixbench-2d12:10:252
1701999230,13cyclictest0-21swapper/210:20:122
17015992314,3cyclictest0-21swapper/111:35:191
17015992314,3cyclictest0-21swapper/107:20:171
1701599231,19cyclictest24504-21/usr/sbin/munin09:50:181
1701599231,17cyclictest0-21swapper/110:01:541
1701599231,13cyclictest25908-21chrt11:07:181
1701599231,11cyclictest0-21swapper/109:41:251
1701099233,4cyclictest0-21swapper/011:09:170
1701099231,12cyclictest29922-21awk10:05:000
47042221,3sleep00-21swapper/010:20:140
1702599223,10cyclictest30135-21/usr/sbin/munin10:05:133
1702599223,10cyclictest22581-21/usr/sbin/munin12:15:203
17025992216,3cyclictest13359-21kernelversion09:25:173
1701999229,3cyclictest0-21swapper/209:31:332
1701999227,3cyclictest0-21swapper/211:04:322
1701999223,2cyclictest87550irq/128-enp1s0-08:43:062
1701599228,4cyclictest26100-21cat12:20:241
1701599225,9cyclictest7167-21rt-features09:10:231
17015992214,3cyclictest0-21swapper/112:05:591
1701599221,16cyclictest0-21swapper/108:04:241
1701599221,16cyclictest0-21swapper/107:29:121
1701599221,15cyclictest0-21swapper/111:41:541
1701599221,15cyclictest0-21swapper/110:44:471
1701599221,15cyclictest0-21swapper/110:37:161
1701599221,15cyclictest0-21swapper/110:18:291
1701599221,15cyclictest0-21swapper/110:18:291
1701599221,11cyclictest9852-21fschecks_time11:45:151
1701599221,11cyclictest0-21swapper/112:37:181
1701599221,11cyclictest0-21swapper/112:37:181
1701599221,11cyclictest0-21swapper/110:10:571
1701599221,11cyclictest0-21swapper/109:29:521
1701599221,11cyclictest0-21swapper/109:17:051
1701599220,16cyclictest0-21swapper/109:45:141
1701099226,14cyclictest0-21swapper/007:45:000
1701099221,8cyclictest19378-21kworker/0:010:57:150
1701099221,6cyclictest0-21swapper/009:52:370
1701099221,3cyclictest9502-21munin-run08:05:010
1701099221,11cyclictest2490-21kworker/0:111:32:440
1701099220,11cyclictest28346-21perf11:15:000
1702599218,2cyclictest20922-21kworker/u8:108:06:303
1702599212,10cyclictest24497-21/usr/sbin/munin07:25:123
1702599211,6cyclictest0-21swapper/312:29:553
1702599211,11cyclictest579-21systemd-journal11:45:013
17025992110,7cyclictest21874-21grep12:10:263
1701999216,5cyclictest27703-21idleruntime-cro10:00:012
1701999211,9cyclictest0-21swapper/210:33:122
1701999211,6cyclictest0-21swapper/211:24:092
1701599215,13cyclictest0-21swapper/111:00:211
17015992116,3cyclictest3014-21sed07:50:011
1701599211,4cyclictest0-21swapper/112:16:211
17015992113,3cyclictest0-21swapper/107:16:041
1701599211,18cyclictest31722-21tail07:40:161
1701599211,17cyclictest16323-21/usr/sbin/munin09:30:251
1701599211,15cyclictest0-21swapper/109:04:071
1701599211,14cyclictest0-21swapper/107:38:041
1701599211,12cyclictest27888-21memory12:25:171
1701599211,11cyclictest12650-21ntp_states11:50:211
1701599211,11cyclictest10983-21smart_sda08:05:231
1701599211,10cyclictest0-21swapper/111:12:131
1701599210,15cyclictest0-21swapper/108:39:211
1701099212,11cyclictest27303-21/usr/sbin/munin08:45:140
1701099211,6cyclictest0-21swapper/010:40:490
17010992115,3cyclictest19032-21sed10:50:250
1701099211,13cyclictest21682-21kworker/0:111:01:150
1702599208,2cyclictest41-21ksoftirqd/310:10:293
1702599203,8cyclictest22912-21/usr/sbin/munin08:35:113
1702599201,11cyclictest30755-21/usr/sbin/munin11:20:233
1702599201,11cyclictest2789-21/usr/sbin/munin11:30:123
1702599201,11cyclictest25356-21perl12:20:163
1702599201,11cyclictest15355-21/usr/sbin/munin10:45:113
1702599201,11cyclictest0-21swapper/312:00:293
1702599201,11cyclictest0-21swapper/309:50:173
1702599201,10cyclictest11313-21diskstats10:35:143
1702599200,8cyclictest0-21swapper/312:39:523
1702599200,8cyclictest0-21swapper/312:39:523
1701999208,3cyclictest0-21swapper/209:16:052
1701999206,8cyclictest0-21swapper/210:51:442
1701999204,9cyclictest755-21rngd07:16:162
1701999201,3cyclictest0-21swapper/211:36:302
1701999201,10cyclictest0-21swapper/210:15:162
1701999201,10cyclictest0-21swapper/210:15:162
1701999200,9cyclictest0-21swapper/209:10:072
17015992015,3cyclictest9103-21df_inode10:30:121
17015992013,3cyclictest0-21swapper/107:58:541
1701599201,14cyclictest0-21swapper/108:52:141
1701599201,14cyclictest0-21swapper/108:11:371
17015992011,3cyclictest0-21swapper/107:51:251
1701599201,11cyclictest17914-21chrt09:35:191
1701599201,11cyclictest16183-21cstates12:00:101
1701599201,11cyclictest0-21swapper/110:47:041
1701599201,10cyclictest761-21unixbench_multi08:55:241
1701599201,10cyclictest22805-21cat08:35:011
1701599201,10cyclictest18230-21runrttasks08:24:231
1701599201,10cyclictest0-21swapper/111:23:161
1701599200,14cyclictest0-21swapper/109:55:021
1701599200,14cyclictest0-21swapper/107:10:031
1701099208,2cyclictest20078-21kworker/0:112:14:320
1701099206,4cyclictest18722chrt10:14:150
1701099201,9cyclictest32633-21kworker/0:011:27:560
1701099201,9cyclictest32633-21kworker/0:011:27:560
1701099201,9cyclictest30398-21kworker/0:211:22:040
1701099201,7cyclictest13348-21kworker/0:111:58:180
1701099201,6cyclictest0-21swapper/011:38:240
1701099201,6cyclictest0-21swapper/009:30:350
304012191,2sleep30-21swapper/312:30:233
1702599196,6cyclictest0-21swapper/311:51:063
1702599193,3cyclictest0-21swapper/307:11:193
1702599191,9cyclictest0-21swapper/307:35:203
1702599191,8cyclictest0-21swapper/312:08:123
1702599191,3cyclictest0-21swapper/310:43:243
1702599191,13cyclictest0-21swapper/308:32:003
1702599191,12cyclictest0-21swapper/309:05:153
1702599191,12cyclictest0-21swapper/308:15:223
1702599191,10cyclictest41-21ksoftirqd/309:15:203
1702599191,10cyclictest21954-21/usr/sbin/munin11:00:203
1702599191,10cyclictest21800-21NetworkManager07:16:373
1702599191,10cyclictest21333-21/usr/sbin/munin09:45:113
1702599191,10cyclictest13776-21/usr/sbin/munin11:55:273
1702599191,10cyclictest0-21swapper/311:25:313
1702599191,10cyclictest0-21swapper/311:25:313
1701999198,3cyclictest200462expr10:55:122
1701999196,4cyclictest0-21swapper/208:04:242
1701999194,3cyclictest0-21swapper/211:53:252
1701999194,3cyclictest0-21swapper/209:37:052
1701999193,9cyclictest0-21swapper/207:13:042
1701999192,3cyclictest0-21swapper/208:34:022
1701999191,8cyclictest3515-21sh09:05:002
1701999191,3cyclictest0-21swapper/212:35:442
1701999191,3cyclictest0-21swapper/212:35:442
1701999191,16cyclictest0-21swapper/207:45:202
1701999191,11cyclictest0-21swapper/210:44:362
1701999190,11cyclictest0-21swapper/210:08:002
1701599191,2cyclictest5717-21diskmemload10:55:031
1701599191,14cyclictest0-21swapper/111:26:141
1701599191,14cyclictest0-21swapper/111:26:141
1701599191,14cyclictest0-21swapper/110:07:251
1701599191,11cyclictest18578-21netstat10:50:191
1701599191,10cyclictest30512-21threads12:30:231
1701099199,8cyclictest17195-21sshd10:48:240
1701099198,3cyclictest1520-21/usr/sbin/munin09:00:250
1701099197,8cyclictest1-21systemd09:09:250
1701099192,4cyclictest18294-21munin-run08:25:020
1701099191,8cyclictest0-21swapper/010:31:230
1701099191,6cyclictest0-21swapper/009:36:500
1701099191,5cyclictest0-21swapper/009:41:590
17010991914,3cyclictest7231-21tail10:25:150
17010991913,3cyclictest3508-21awk10:15:230
17010991913,3cyclictest3508-21awk10:15:230
17010991913,3cyclictest27347-21sed07:30:150
1701099190,9cyclictest32284-21kworker/0:007:16:450
1701099190,12cyclictest11690-21timerandwakeup09:20:230
308102181,3sleep00-21swapper/007:39:290
261042181,2sleep00-21swapper/007:25:280
203372181,3sleep30-21swapper/308:28:163
178642181,14sleep25717-21diskmemload12:03:582
1702599188,3cyclictest0-21swapper/309:40:163
1702599187,3cyclictest7464-21latency10:25:163
1702599182,7cyclictest0-21swapper/311:09:093
1702599182,6cyclictest0-21swapper/310:30:113
17025991814,3cyclictest0-21swapper/308:49:273
17025991812,3cyclictest23412-21ls07:20:223
1702599181,15cyclictest0-21swapper/307:45:083
1702599181,11cyclictest0-21swapper/311:10:133
1702599181,11cyclictest0-21swapper/310:15:123
1702599181,11cyclictest0-21swapper/310:15:123
1702599180,6cyclictest0-21swapper/310:02:003
1701999188,8cyclictest0-21swapper/211:17:292
1701999183,3cyclictest87350irq/126-enp1s0-11:55:152
1701999182,8cyclictest0-21swapper/208:48:502
1701999181,3cyclictest0-21swapper/212:08:532
1701999181,3cyclictest0-21swapper/210:26:492
17019991813,3cyclictest5670-21diskstats07:55:142
17019991813,2cyclictest311ktimersoftd/208:52:342
1701999181,2cyclictest0-21swapper/209:54:172
1701999181,14cyclictest23546-21head08:35:142
1701999181,11cyclictest0-21swapper/211:40:412
1701599181,3cyclictest0-21swapper/108:40:411
1701599180,2cyclictest0-21swapper/108:46:141
1701099188,3cyclictest5945-21/usr/sbin/munin09:10:120
1701099185,2cyclictest20453-21kworker/0:308:34:550
1701099184,8cyclictest0-21swapper/012:29:140
1701099181,5cyclictest0-21swapper/009:26:230
1701099181,4cyclictest0-21swapper/011:44:180
17010991811,4cyclictest22883-21grep09:45:240
1701099181,10cyclictest808-21snmpd07:57:180
1702599175,4cyclictest0-21swapper/309:13:583
1702599171,8cyclictest11530-21processes09:20:213
1702599171,7cyclictest26695-21/usr/sbin/munin07:30:223
1702599171,4cyclictest5072-21idleruntime10:20:173
1702599171,11cyclictest0-21swapper/310:50:183
1701999175,2cyclictest87650irq/129-enp1s0-07:40:412
1701999171,8cyclictest4299-21sed11:30:252
1701999171,2cyclictest87350irq/126-enp1s0-07:38:572
17019991712,3cyclictest11046-21sort09:20:162
1701999171,1cyclictest0-21swapper/208:12:182
1701999171,13cyclictest0-21swapper/208:19:022
1701999171,10cyclictest0-21swapper/207:23:452
1701999170,5cyclictest0-21swapper/209:26:532
1701599171,15cyclictest0-21swapper/108:15:221
1701099179,5cyclictest13456-21sed08:10:260
1701099172,12cyclictest25106-21/usr/sbin/munin08:40:190
1701099171,6cyclictest0-21swapper/009:59:510
1701099171,5cyclictest0-21swapper/012:15:050
17010991712,3cyclictest1058-21seq08:57:140
1701099171,13cyclictest9208-21ntp_states09:15:210
1701099171,13cyclictest3122-21/usr/sbin/munin07:50:100
1701099171,10cyclictest0-21swapper/010:05:470
266852161,2sleep20-21swapper/207:30:052
1702599161,8cyclictest0-21swapper/311:35:183
1702599161,6cyclictest0-21swapper/308:50:083
17025991612,2cyclictest39-21rcuc/308:20:123
17025991611,3cyclictest32396-21sed07:40:253
17025991611,3cyclictest11922-21/usr/sbin/munin08:10:253
17025991610,3cyclictest3332-21expr07:50:123
1701999161,3cyclictest0-21swapper/211:45:232
17019991612,3cyclictest0-21swapper/208:28:122
1701999161,14cyclictest0-21swapper/207:27:262
17019991610,3cyclictest3514-21latency_hist09:05:022
17019991610,3cyclictest32050-21entropy08:55:122
1701099167,3cyclictest14205-1kworker/u9:112:00:120
1701099161,6cyclictest12552-21users10:35:280
17010991611,3cyclictest871-21date07:45:010
17010991611,3cyclictest22891-21gltestperf07:20:150
17025991510,3cyclictest7519-21/usr/sbin/munin08:00:133
1702599143,2cyclictest87650irq/129-enp1s0-09:00:123
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional