You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-30 - 19:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Thu Jul 30, 2026 12:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15523226697,24sleep20-21swapper/207:09:162
1585799221211,8cyclictest18411-21lspci08:27:562
154822164101,34sleep30-21swapper/307:08:443
153572143107,25sleep10-21swapper/107:07:051
154072138102,25sleep00-21swapper/007:07:440
119042610,2sleep30-21swapper/311:15:223
98902570,8sleep228-21ksoftirqd/209:14:172
257732520,3chrt25770-21ssh09:25:223
250972520,2sleep10-21swapper/108:40:261
306212500,2sleep334-21ksoftirqd/310:41:023
231022480,2sleep334-21ksoftirqd/309:47:383
184042470,1sleep30-21swapper/308:27:573
65532450,2sleep00-21swapper/012:00:140
197842450,2chrt0-21swapper/210:33:402
124892450,1sleep20-21swapper/210:28:062
228212440,1chrt0-21swapper/112:11:301
162442440,4chrt16242-21ssh10:30:243
57282420,2chrt0-21swapper/111:59:501
285762400,8sleep0131rcuc/010:40:120
237372400,3chrt0-21swapper/109:48:161
274222390,1sleep20-21swapper/210:15:142
228432310,2chrt0-21swapper/108:35:241
192112310,4sleep3341ksoftirqd/311:20:323
192112310,4sleep3341ksoftirqd/311:20:313
1585799307,8cyclictest28-21ksoftirqd/212:05:142
15857993012,11cyclictest28-21ksoftirqd/210:06:352
15857993011,12cyclictest28-21ksoftirqd/207:10:332
262322290,2sleep30-21swapper/311:25:503
1585799298,7cyclictest28-21ksoftirqd/211:05:222
15857992911,4cyclictest28-21ksoftirqd/209:45:012
15857992910,9cyclictest28-21ksoftirqd/209:37:222
15855992911,8cyclictest12-21ksoftirqd/011:06:380
15855992911,5cyclictest12-21ksoftirqd/007:31:030
264292280,7sleep228-21ksoftirqd/210:38:372
15855992813,6cyclictest12-21ksoftirqd/008:57:100
15855992810,9cyclictest12-21ksoftirqd/012:33:200
15855992810,9cyclictest12-21ksoftirqd/012:19:020
1585799277,11cyclictest28-21ksoftirqd/212:16:522
1585799276,10cyclictest28-21ksoftirqd/209:25:342
1585799272,6cyclictest28-21ksoftirqd/209:22:452
1585799271,7cyclictest28-21ksoftirqd/211:35:002
1585799271,7cyclictest28-21ksoftirqd/209:15:252
15857992712,3cyclictest28-21ksoftirqd/210:13:452
1585699271,24cyclictest0-21swapper/107:12:001
1585599279,11cyclictest12-21ksoftirqd/011:28:140
1585599272,7cyclictest12-21ksoftirqd/010:25:200
15855992717,8cyclictest9289-21ssh10:01:040
15855992710,8cyclictest12-21ksoftirqd/007:38:160
15858992612,5cyclictest13380-21date11:40:253
1585799267,11cyclictest28-21ksoftirqd/212:12:332
1585799267,10cyclictest28-21ksoftirqd/209:53:292
1585799266,10cyclictest28-21ksoftirqd/209:57:112
1585799265,9cyclictest28-21ksoftirqd/212:04:102
1585799262,4cyclictest281ksoftirqd/210:46:132
15857992622,3cyclictest271rcuc/211:27:122
15856992610,4cyclictest131042sh11:16:101
1585599269,9cyclictest141rcu_preempt09:26:030
1585599267,10cyclictest12-21ksoftirqd/009:15:040
1585599266,4cyclictest12-21ksoftirqd/010:53:290
1585599266,11cyclictest12-21ksoftirqd/011:45:250
1585599264,10cyclictest12-21ksoftirqd/012:20:140
15855992623,2cyclictest131rcuc/011:11:270
15855992623,2cyclictest0-21swapper/007:11:520
15855992621,3cyclictest131rcuc/012:13:570
1585599261,9cyclictest5909-21munin-run11:59:590
1585599260,6cyclictest12-21ksoftirqd/011:20:010
1585599260,6cyclictest12-21ksoftirqd/008:28:300
276332250,2chrt0-21swapper/311:51:023
1585899251,23cyclictest0-21swapper/307:12:013
1585799259,6cyclictest28-21ksoftirqd/212:25:592
1585799257,10cyclictest28-21ksoftirqd/211:40:522
1585799255,11cyclictest28-21ksoftirqd/207:58:322
1585799254,7cyclictest28-21ksoftirqd/210:01:082
1585799254,11cyclictest28-21ksoftirqd/210:44:482
1585799252,7cyclictest28-21ksoftirqd/211:58:012
15857992511,3cyclictest28-21ksoftirqd/211:04:192
15857992510,4cyclictest28-21ksoftirqd/211:15:202
1585699251,9cyclictest20838-21fschecks_time10:10:161
15856992514,8cyclictest19657-21df_inode08:30:161
1585599259,8cyclictest12-21ksoftirqd/010:59:090
1585599257,8cyclictest12-21ksoftirqd/009:47:010
1585599257,7cyclictest12-21ksoftirqd/010:45:220
1585599256,10cyclictest12-21ksoftirqd/010:15:220
1585599254,7cyclictest12-21ksoftirqd/009:20:160
1585599252,8cyclictest12-21ksoftirqd/009:10:120
1585599251,5cyclictest12-21ksoftirqd/011:35:220
1585599250,6cyclictest11884-21apt-get11:40:130
297512240,2sleep122-21ksoftirqd/107:40:151
15858992415,7cyclictest19219-21ssh10:57:003
1585799249,6cyclictest28-21ksoftirqd/211:11:582
1585799248,7cyclictest28-21ksoftirqd/211:52:092
1585799247,7cyclictest28-21ksoftirqd/212:37:262
1585799246,7cyclictest28-21ksoftirqd/211:46:122
1585799246,10cyclictest28-21ksoftirqd/212:31:452
1585799246,10cyclictest28-21ksoftirqd/210:55:302
1585799245,11cyclictest31137-21runrttasks07:44:322
1585799245,10cyclictest28-21ksoftirqd/210:20:412
1585799244,12cyclictest28-21ksoftirqd/208:19:192
1585799244,12cyclictest28-21ksoftirqd/208:19:182
1585799244,12cyclictest28-21ksoftirqd/207:25:002
15857992420,3cyclictest271rcuc/209:30:122
15857992419,3cyclictest271rcuc/208:56:502
1585799241,8cyclictest28-21ksoftirqd/209:47:272
15857992418,4cyclictest28-21ksoftirqd/211:35:012
15857992418,4cyclictest28-21ksoftirqd/211:25:012
15857992418,4cyclictest28-21ksoftirqd/211:25:002
15857992410,3cyclictest28-21ksoftirqd/212:23:002
15856992414,6cyclictest53050irq/142-enp3s0-11:35:111
1585599249,7cyclictest12-21ksoftirqd/011:01:130
1585599249,3cyclictest141rcu_preempt08:04:520
1585599248,9cyclictest12-21ksoftirqd/010:12:480
1585599247,8cyclictest141rcu_preempt09:36:460
1585599246,9cyclictest12-21ksoftirqd/009:51:570
1585599244,9cyclictest12-21ksoftirqd/009:55:530
1585599244,8cyclictest12-21ksoftirqd/012:25:090
1585599242,5cyclictest6541-21rm10:23:520
15855992419,3cyclictest131rcuc/008:45:150
15855992410,3cyclictest12-21ksoftirqd/010:06:250
1585599240,8cyclictest12-21ksoftirqd/012:36:300
1585599240,8cyclictest12-21ksoftirqd/011:51:470
1585899232,8cyclictest1377-21apt-get10:20:113
15858992314,6cyclictest12159-21cut09:15:213
1585899231,12cyclictest2770-21users12:20:243
1585799236,10cyclictest774-21runrttasks07:48:322
1585799235,9cyclictest1654-21runrttasks09:04:242
1585799235,11cyclictest28-21ksoftirqd/207:15:112
1585799233,6cyclictest28-21ksoftirqd/210:51:212
1585799231,9cyclictest28-21ksoftirqd/207:30:122
15857992316,5cyclictest28-21ksoftirqd/207:25:002
1585699231,8cyclictest29672-21expr10:40:221
1585699231,6cyclictest8479-21diskstats10:25:141
1585599239,2cyclictest141rcu_preempt07:23:520
1585599236,8cyclictest141rcu_preempt08:07:180
1585599236,3cyclictest141rcu_preempt09:06:130
1585599236,3cyclictest141rcu_preempt09:06:120
1585599236,10cyclictest12-21ksoftirqd/010:36:240
1585599233,8cyclictest12-21ksoftirqd/009:40:130
1585599233,12cyclictest12-21ksoftirqd/009:31:300
1585599231,8cyclictest12-21ksoftirqd/010:30:250
15855992318,3cyclictest131rcuc/007:50:180
15855992313,8cyclictest131rcuc/012:05:490
15855992310,4cyclictest12-21ksoftirqd/011:32:470
272662220,4sleep10-21swapper/111:50:401
1585899226,14cyclictest7362-21idleruntime-cro08:04:593
1585899224,9cyclictest141rcu_preempt10:49:113
15858992217,4cyclictest34-21ksoftirqd/310:00:013
1585899221,6cyclictest14001-21diskstats10:05:143
1585799225,9cyclictest0-21swapper/209:05:532
1585799225,9cyclictest0-21swapper/209:05:522
1585799224,7cyclictest28-21ksoftirqd/208:00:222
1585799224,10cyclictest28-21ksoftirqd/208:36:482
1585799222,3cyclictest28-21ksoftirqd/207:35:192
15857992217,3cyclictest271rcuc/208:47:512
1585699221,6cyclictest22074-21memory12:35:171
15856992214,5cyclictest956-21sh10:20:001
15856992213,7cyclictest4603-21ls07:55:241
15856992211,7cyclictest53050irq/142-enp3s0-09:10:131
1585599228,3cyclictest141rcu_preempt08:54:530
1585599226,7cyclictest12-21ksoftirqd/008:34:510
1585599224,6cyclictest12-21ksoftirqd/007:15:210
1585599223,7cyclictest12-21ksoftirqd/007:29:240
1585599221,6cyclictest3296-21apt-get07:55:130
15855992212,7cyclictest13154-21cat08:15:240
15855992212,7cyclictest13154-21cat08:15:230
1585599220,7cyclictest12-21ksoftirqd/011:22:200
1585599220,7cyclictest12-21ksoftirqd/011:22:190
1585899214,3cyclictest3856-21diskmemload12:36:553
1585899214,3cyclictest3856-21diskmemload11:56:513
1585899212,17cyclictest971-21run-parts10:20:003
1585899212,14cyclictest31247-21ssh09:30:143
1585899211,5cyclictest4537-21hddtemp_smartct11:10:153
15858992113,6cyclictest32347-21ntpq07:45:203
1585699214,3cyclictest3856-21diskmemload10:45:561
1585699213,5cyclictest22-21ksoftirqd/108:10:121
1585699213,5cyclictest141rcu_preempt12:15:141
1585699211,5cyclictest14144-21ntp_states12:05:221
15856992114,5cyclictest1173-21cpu07:50:111
15856992113,6cyclictest25114-21ntpq11:25:191
15856992113,6cyclictest13412-21needreboot09:40:191
15856992112,6cyclictest31314-21unixbench_singl11:05:261
15856992110,9cyclictest13681-21sh11:40:311
1585599219,2cyclictest141rcu_preempt08:35:060
1585599218,2cyclictest141rcu_preempt08:22:260
15855992115,4cyclictest131rcuc/009:00:210
15855992110,2cyclictest141rcu_preempt07:45:440
36162200,2sleep30-21swapper/307:55:133
243762200,2sleep10-21swapper/109:25:081
1585899205,2cyclictest141rcu_preempt12:30:243
1585899204,3cyclictest3856-21diskmemload11:09:423
1585899204,3cyclictest3856-21diskmemload09:36:033
1585899204,14cyclictest7928-21cstates12:25:113
1585899203,3cyclictest3856-21diskmemload09:42:183
1585899201,6cyclictest18210-21ssh09:20:183
15858992015,3cyclictest11625-21sh08:15:013
15858992012,6cyclictest27041-21apt-get07:35:123
1585899201,16cyclictest21918-21fschecks_time08:35:153
15858992011,6cyclictest20941-21tail10:10:163
15858992011,6cyclictest20908-21cat07:20:183
1585899200,18cyclictest20422-21which08:30:233
1585799204,8cyclictest28-21ksoftirqd/208:31:132
1585799203,10cyclictest25221-21runrttasks08:41:252
15857992014,4cyclictest8181-21idleruntime08:05:172
1585699208,10cyclictest5171-21ssh12:22:481
1585699204,3cyclictest3856-21diskmemload11:01:511
1585699203,3cyclictest3856-21diskmemload09:30:071
1585699203,3cyclictest3856-21diskmemload09:21:131
1585699203,11cyclictest75-21kswapd010:30:101
1585699202,13cyclictest3920-21ssh11:10:121
1585699201,6cyclictest9077-21kworker/u8:110:10:001
15856992013,5cyclictest2525-21smartctl09:05:161
15856992013,5cyclictest2525-21smartctl09:05:151
15856992013,4cyclictest636-21runrttasks12:02:361
15856992012,5cyclictest22273-21sendmail11:00:001
1585599206,3cyclictest141rcu_preempt07:44:350
1585599202,6cyclictest12-21ksoftirqd/008:42:560
15855992011,6cyclictest11094-21cat08:10:240
1585899196,4cyclictest141rcu_preempt11:35:153
1585899195,3cyclictest3856-21diskmemload10:36:003
1585899194,3cyclictest141rcu_preempt11:30:113
1585899193,4cyclictest34-21ksoftirqd/310:25:113
15858991914,3cyclictest24133-21gltestperf08:40:173
15858991912,5cyclictest31234-21seq12:18:123
15858991912,5cyclictest13675-21seq08:19:113
15858991912,5cyclictest13675-21seq08:19:103
15858991911,6cyclictest8030-21perl08:05:143
15858991911,6cyclictest517-21cat09:00:173
15858991911,6cyclictest29366-21awk08:50:283
15858991911,6cyclictest23125-21sed07:25:153
15858991911,5cyclictest52950irq/141-enp3s0-11:45:093
15858991910,6cyclictest31139-21readlink08:55:233
15858991910,6cyclictest14635-21sed12:05:243
1585799197,2cyclictest141rcu_preempt07:54:192
1585699198,9cyclictest53050irq/142-enp3s0-11:46:121
1585699192,3cyclictest3856-21diskmemload10:01:551
1585699191,5cyclictest636-21runrttasks10:35:031
15856991914,3cyclictest2164-21grep10:20:181
15856991912,5cyclictest7477-21cut08:05:011
15856991912,4cyclictest17488-21tr11:20:111
15856991912,4cyclictest17488-21tr11:20:101
15856991911,6cyclictest7170-21cat09:35:251
15856991911,6cyclictest15964-21cut08:25:001
15856991911,6cyclictest15336-21seq09:18:001
15856991911,6cyclictest10317-21ntpq10:50:221
15856991910,6cyclictest21435-21date07:20:231
1585899189,6cyclictest560-21dbus-daemon12:10:003
1585899186,4cyclictest52950irq/141-enp3s0-10:50:223
1585899183,13cyclictest15002-21cut08:20:213
1585899182,14cyclictest19014-21cut07:15:213
1585899181,15cyclictest8185-21sensors_temp10:00:203
15858991810,6cyclictest6627-21expr12:00:143
15858991810,6cyclictest29411-21grep07:40:123
15858991810,6cyclictest1468-21expr07:50:113
15858991810,5cyclictest5206-21sed09:10:203
15858991810,5cyclictest25973-21timerandwakeup07:30:243
1585799184,2cyclictest141rcu_preempt08:10:132
1585799181,5cyclictest28-21ksoftirqd/208:20:212
1585799180,7cyclictest28-21ksoftirqd/208:50:262
1585699189,7cyclictest536-21df_inode09:55:141
1585699187,9cyclictest53050irq/142-enp3s0-12:33:161
1585699182,14cyclictest8323-21tr12:25:141
1585699182,11cyclictest22553-21apt-get07:25:121
1585699181,6cyclictest16316-21cut08:25:101
15856991812,4cyclictest30785-21/usr/sbin/munin11:30:151
15856991811,5cyclictest698-21open_files09:00:181
15856991811,5cyclictest31806-21seq08:58:431
15856991811,5cyclictest26149-21uniq08:45:171
1585699181,14cyclictest18151-21cpuspeed_turbos07:15:131
15856991810,6cyclictest27279-21cat07:35:141
15856991810,6cyclictest26758-21expr09:50:181
1585899171,4cyclictest0-21swapper/309:50:033
15858991710,5cyclictest22763-21grep11:00:113
1585699179,6cyclictest24777-21grep07:30:121
1585699173,4cyclictest130832switchtime08:15:231
1585699173,4cyclictest130832switchtime08:15:231
15856991710,5cyclictest28816-21cat08:50:211
15858991611,3cyclictest3062-21mailstats09:05:213
15858991611,3cyclictest3062-21mailstats09:05:213
15858991611,3cyclictest25751-21acpi08:45:133
1585699169,5cyclictest31522-21sort07:45:121
15856991611,3cyclictest7642-21cpu08:05:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional