You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-01 - 21:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Wed Apr 01, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1196429582,9sleep20-21swapper/207:08:422
1171928660,9sleep00-21swapper/007:05:320
1196327857,6sleep10-21swapper/107:08:411
1177927661,9sleep30-21swapper/307:06:183
136512720,2sleep232-21ksoftirqd/209:34:512
1214599314,26cyclictest25800-21df_abs10:20:180
1214599312,27cyclictest6709-21munin-run09:50:000
1214599302,27cyclictest4738-21diskmemload12:09:240
1214599302,27cyclictest4738-21diskmemload10:41:550
1214599291,27cyclictest4738-21diskmemload11:45:060
1214599291,27cyclictest4738-21diskmemload09:24:130
1214599290,28cyclictest4738-21diskmemload10:46:120
1214599290,28cyclictest0-21swapper/009:41:250
1214599290,1cyclictest0-21swapper/007:20:140
1214599282,25cyclictest0-21swapper/012:35:210
1214599280,25cyclictest0-21swapper/010:00:210
1214599280,25cyclictest0-21swapper/007:40:120
1214599272,24cyclictest4738-21diskmemload11:56:360
1214599271,22cyclictest0-21swapper/011:17:550
1214599271,22cyclictest0-21swapper/008:20:000
1214599271,20cyclictest4738-21diskmemload12:30:100
1214599270,26cyclictest0-21swapper/010:10:000
1214599270,25cyclictest0-21swapper/010:25:180
1214599270,0cyclictest0-21swapper/010:53:510
1214599263,22cyclictest4738-21diskmemload11:31:470
1214599261,24cyclictest4738-21diskmemload11:54:140
1214599261,19cyclictest4738-21diskmemload09:36:410
1214599260,20cyclictest11966-21chrt09:52:360
1214599260,1cyclictest0-21swapper/011:10:550
1215199253,20cyclictest32-21ksoftirqd/210:16:032
1214599253,21cyclictest4738-21diskmemload09:13:500
1214599250,3cyclictest4738-21diskmemload11:03:110
1214599250,3cyclictest0-21swapper/012:02:000
1214599250,1cyclictest0-21swapper/009:33:030
239682240,0sleep20-21swapper/211:55:402
1215199240,24cyclictest0-21swapper/211:32:462
1214999240,21cyclictest0-21swapper/110:10:091
1214599242,21cyclictest9753-21pmu-power10:10:220
1214599241,23cyclictest4738-21diskmemload12:11:290
1214599241,19cyclictest3873-21grep09:05:210
1214599240,23cyclictest0-21swapper/011:20:140
1214599240,23cyclictest0-21swapper/010:34:530
1214599240,1cyclictest0-21swapper/011:07:320
1214599233,19cyclictest4738-21diskmemload09:28:140
1214599231,20cyclictest4738-21diskmemload10:59:310
1214599230,2cyclictest0-21swapper/011:36:470
1214599230,22cyclictest4738-21diskmemload09:17:100
1214599230,22cyclictest4738-21diskmemload09:17:100
1214599230,1cyclictest0-21swapper/011:42:510
1214599230,19cyclictest23651-21grep07:30:270
1215599221,21cyclictest0-21swapper/311:36:123
12155992210,0cyclictest141rcu_preempt11:44:193
1215599220,17cyclictest21943-21ssh09:20:143
1214599222,20cyclictest0-21swapper/010:38:300
1214599222,14cyclictest0-21swapper/009:55:330
1214599220,21cyclictest0-21swapper/010:15:340
1214599220,1cyclictest4738-21diskmemload11:29:530
1214599220,1cyclictest4738-21diskmemload11:29:530
12151992120,1cyclictest311rcuc/212:11:422
12151992120,1cyclictest311rcuc/209:19:002
12151992120,1cyclictest311rcuc/209:18:592
12151992118,2cyclictest32-21ksoftirqd/212:40:012
1215199211,2cyclictest141rcu_preempt09:43:392
1214999218,8cyclictest14577-21apt-get09:35:121
1214599211,19cyclictest4738-21diskmemload12:17:090
1214599211,19cyclictest20451-21if_enp0s31f607:25:180
1214599211,19cyclictest0-21swapper/008:35:150
1214599210,2cyclictest0-21swapper/012:27:470
1214599210,19cyclictest0-21swapper/008:10:180
12155992020,0cyclictest0-21swapper/311:24:243
1215599200,4cyclictest0-21swapper/310:55:183
1215199201,18cyclictest0-21swapper/209:38:492
1215199201,17cyclictest13626-21cat11:50:152
1214999209,10cyclictest27429-21sh10:01:351
12149992013,5cyclictest12770-21perf11:50:001
1214599202,14cyclictest32565-21ssh12:20:210
1214599201,18cyclictest24024-21if_err_enp0s31f08:40:160
1214599200,0cyclictest0-21swapper/009:00:140
12155991918,1cyclictest0-21swapper/309:56:563
1215599191,1cyclictest0-21swapper/311:01:203
1215599190,4cyclictest0-21swapper/310:25:243
1215199198,6cyclictest32-21ksoftirqd/210:39:472
12151991917,1cyclictest6713-21awk09:50:002
1215199190,17cyclictest0-21swapper/211:38:272
1214599193,15cyclictest24349-21apt-get07:35:150
1214599190,1cyclictest17147-21ntpq08:25:210
1215599188,1cyclictest141rcu_preempt09:44:573
1215599181,0cyclictest0-21swapper/309:36:513
1215599180,17cyclictest0-21swapper/309:54:503
1215199186,6cyclictest4738-21diskmemload10:10:222
12151991816,1cyclictest32-21ksoftirqd/211:24:232
1215199181,16cyclictest32-21ksoftirqd/211:25:072
1215199181,16cyclictest32-21ksoftirqd/211:25:072
12151991810,6cyclictest4738-21diskmemload11:05:492
12149991817,1cyclictest0-21swapper/110:31:131
1214999180,18cyclictest0-21swapper/112:32:421
1214999180,18cyclictest0-21swapper/112:29:021
1214999180,18cyclictest0-21swapper/112:24:321
1214999180,18cyclictest0-21swapper/110:37:121
1214599180,17cyclictest0-21swapper/007:50:000
1214599180,15cyclictest0-21swapper/007:50:160
12151991712,5cyclictest311rcuc/207:25:192
12151991710,1cyclictest4953-21ssh11:45:142
12149991716,1cyclictest0-21swapper/112:11:121
12155991612,3cyclictest141rcu_preempt10:51:473
1215199167,6cyclictest32-21ksoftirqd/211:00:002
1215199165,10cyclictest19621-21/usr/sbin/munin11:15:112
1215199164,1cyclictest311rcuc/210:51:542
12151991611,5cyclictest311rcuc/212:00:472
12151991610,6cyclictest0-21swapper/210:25:212
1215199160,2cyclictest0-21swapper/209:10:522
12149991616,0cyclictest0-21swapper/107:49:421
1214999160,16cyclictest0-21swapper/112:35:451
1214599168,7cyclictest5454-21grep08:00:260
1214599168,7cyclictest31202-21grep08:55:200
1214599168,7cyclictest2515-21sed07:55:200
1214599161,14cyclictest0-21swapper/008:45:140
1214599160,15cyclictest0-21swapper/008:05:230
1215599157,3cyclictest39-21ksoftirqd/312:35:153
1215599151,2cyclictest141rcu_preempt10:11:043
1215599150,2cyclictest0-21swapper/312:23:313
1215599150,13cyclictest13117-21runrttasks12:28:293
1215199153,1cyclictest2756050kworker/u8:010:30:162
12151991515,0cyclictest0-21swapper/210:20:122
12151991513,1cyclictest4738-21diskmemload12:07:142
12151991513,1cyclictest0-21swapper/207:20:142
12151991512,3cyclictest311rcuc/210:45:472
12151991512,1cyclictest0-21swapper/209:25:002
12151991510,5cyclictest17009-21apache210:09:572
12149991514,1cyclictest0-21swapper/109:29:581
12149991513,2cyclictest25-21ksoftirqd/109:30:171
12149991510,3cyclictest30497-21tr08:55:151
226672140,1chrt0-21swapper/112:15:021
1215599141,7cyclictest0-21swapper/311:30:023
12155991412,1cyclictest39-21ksoftirqd/310:46:343
12155991411,2cyclictest39-21ksoftirqd/312:15:003
12155991410,3cyclictest14802-21grep08:20:223
1215599140,1cyclictest0-21swapper/309:32:283
1215199147,4cyclictest32-21ksoftirqd/209:58:532
1215199143,9cyclictest0-21swapper/211:04:022
1215199140,1cyclictest0-21swapper/211:40:212
1214999141,9cyclictest0-21swapper/109:15:131
1214999141,9cyclictest0-21swapper/109:15:131
12149991414,0cyclictest0-21swapper/110:55:141
12149991413,1cyclictest4738-21diskmemload11:04:421
1214999141,13cyclictest29044-21ssh11:59:391
1214999141,11cyclictest0-21swapper/109:10:141
12149991410,3cyclictest27213-21cat10:40:191
1214999140,1cyclictest0-21swapper/108:50:131
1214999140,12cyclictest9113-21wc10:30:001
1214999140,0cyclictest0-21swapper/110:45:151
1214599147,6cyclictest29868-21chrt08:53:250
1214599140,13cyclictest0-21swapper/008:30:160
1215599139,3cyclictest24605-21cron10:20:013
1215599137,3cyclictest39-21ksoftirqd/312:35:003
12155991311,1cyclictest39-21ksoftirqd/311:06:113
12155991310,2cyclictest5686-21cat11:25:273
12155991310,2cyclictest5686-21cat11:25:263
12155991310,2cyclictest27964-21grep07:40:203
12155991310,2cyclictest26046-21grep07:35:293
12155991310,2cyclictest23030-21chrt10:37:593
12155991310,2cyclictest21727-21basename07:30:003
1215599130,2cyclictest141rcu_preempt11:12:313
1215599130,13cyclictest0-21swapper/310:34:133
1215599130,0cyclictest141rcu_preempt10:06:383
1215199139,3cyclictest4738-21diskmemload10:40:162
1215199139,3cyclictest15516-21head07:15:172
1215199136,6cyclictest32-21ksoftirqd/210:01:342
12151991311,1cyclictest311rcuc/212:18:012
12151991311,1cyclictest26740-21apt-get07:40:122
12151991310,2cyclictest8288-21sort09:50:192
12151991310,2cyclictest14101-21ssh11:11:042
1215199130,12cyclictest30070-21cat08:55:002
1214999139,3cyclictest24705-21ssh10:20:001
1214999139,2cyclictest18260-21wc08:30:001
12149991311,1cyclictest12790-21ls11:30:161
12149991310,2cyclictest28895-21grep11:20:171
12149991310,2cyclictest21279-21grep07:25:271
12149991310,2cyclictest16952-21ls09:55:221
1214999130,1cyclictest0-21swapper/111:50:201
1214599137,5cyclictest13703-21wc07:10:220
1214599130,1cyclictest0-21swapper/008:20:140
1215599129,2cyclictest30006-21munin-run08:55:003
1215599129,2cyclictest27416-21ntpq10:40:203
1215599129,2cyclictest25382-21grep10:00:223
1215599129,2cyclictest21840-21grep08:35:203
1215599129,2cyclictest14123-21grep11:50:183
1215599129,2cyclictest13024-21ssh09:15:003
1215599128,3cyclictest6426-21tr08:05:163
1215599128,3cyclictest29363-21grep07:45:153
1215599128,3cyclictest14070-21ssh09:15:153
1215599128,3cyclictest14070-21ssh09:15:153
1215599124,4cyclictest225-21systemd-journal07:35:003
1215599122,1cyclictest141rcu_preempt11:19:343
12155991210,1cyclictest39-21ksoftirqd/311:55:253
12155991210,1cyclictest39-21ksoftirqd/310:24:023
1215199129,2cyclictest24571-21cut07:35:162
1215199129,2cyclictest0-21swapper/212:21:482
1215199126,2cyclictest27231-21sed08:45:262
1215199124,5cyclictest32-21ksoftirqd/209:25:132
1215199122,2cyclictest141rcu_preempt08:04:492
12151991210,1cyclictest311rcuc/208:16:252
12151991210,1cyclictest29656-21df07:45:152
1214999129,2cyclictest4722-21sh09:10:001
1214999129,2cyclictest3122-21grep07:55:261
1214999129,2cyclictest23870-21head08:40:151
1214999129,2cyclictest21219-21chrt11:35:151
1214999129,2cyclictest15119-21grep09:55:001
1214999128,3cyclictest26118-21ssh09:41:201
1214999128,3cyclictest17960-21head07:20:171
1214999128,1cyclictest14705-21ssh10:52:171
1214999127,3cyclictest22220-21tr09:20:141
12149991210,2cyclictest4738-21diskmemload09:49:081
1214999120,1cyclictest0-21swapper/110:20:161
1214999120,11cyclictest0-21swapper/112:00:191
1214599120,8cyclictest225-21systemd-journal07:19:030
1215599118,2cyclictest9287-21head08:10:163
1215599118,2cyclictest4671-21cat08:00:183
1215599118,2cyclictest3277-21chrt07:56:433
1215599118,2cyclictest32458-21chrt09:25:383
1215599118,2cyclictest32022-21sort09:45:183
1215599118,2cyclictest31041-21tail08:55:193
1215599118,2cyclictest2182-21ssh12:02:273
1215599118,2cyclictest16670-21head08:25:183
1215599118,2cyclictest1497-21wc09:00:223
1215599118,2cyclictest10237-21ssh12:07:143
1215599117,3cyclictest22662-21ssh12:15:023
1215599112,7cyclictest20076-21unixbench_singl08:30:273
1215199119,1cyclictest7500-21grep12:25:152
1215199119,1cyclictest3294-21sort09:05:162
1215199118,2cyclictest32207-21grep07:50:162
1215199115,4cyclictest32-21ksoftirqd/212:30:212
1215199113,5cyclictest32-21ksoftirqd/209:00:132
1215199111,5cyclictest0-21swapper/208:35:182
1215199110,10cyclictest31296-21cat08:55:202
1214999118,2cyclictest8860-21grep08:10:141
1214999118,2cyclictest6060-21grep12:05:141
1214999118,2cyclictest503-21grep09:00:151
1214999118,2cyclictest4248-21grep11:25:151
1214999118,2cyclictest4248-21grep11:25:151
1214999118,2cyclictest4086-21grep08:00:151
1214999118,2cyclictest32279-21ssh10:05:131
1214999118,2cyclictest3078-21grep11:05:161
1214999118,2cyclictest26230-21head08:45:171
1214999118,2cyclictest25680-21ssh11:18:341
1214999118,2cyclictest20633-21cat08:35:001
1214999118,2cyclictest17383-21ssh11:13:361
1214999117,2cyclictest12487-21grep07:10:141
1214999115,6cyclictest17009-21apache207:44:281
1215599108,1cyclictest39-21ksoftirqd/308:45:193
1215599108,1cyclictest19106-21taskset07:22:183
1215599108,1cyclictest15748-21wc07:15:193
1215599107,2cyclictest4743-21ssh11:45:053
1215599107,2cyclictest4112-21cat09:05:243
1215599107,2cyclictest23237-21cut08:40:123
1215599107,2cyclictest12167-21cut08:15:223
1215199108,2cyclictest23122-21acpi08:40:102
1215199108,1cyclictest2530-21cut07:55:202
1215199107,2cyclictest13799-21cat07:10:262
1215199105,4cyclictest13639-21wget08:20:152
1215199100,4cyclictest225-21systemd-journal08:10:182
1214999108,2cyclictest7029-21idleruntime08:05:191
1214999108,1cyclictest29310-21expr11:40:131
1214999108,1cyclictest12726-21chrt08:15:441
1214999107,2cyclictest25467-21tr07:35:221
1214999107,2cyclictest15130-21expr07:15:151
1214999107,2cyclictest14661-21grep08:20:201
1214999100,9cyclictest237542sleep107:31:001
1214999100,8cyclictest0-21swapper/107:50:121
121559997,1cyclictest1119-21taskset07:54:353
121559990,5cyclictest13164-21head07:10:173
121519998,1cyclictest21966-21apache_accesses07:30:122
121519996,2cyclictest6242-21grep08:05:152
121519995,3cyclictest18835-21expr08:30:162
121499996,2cyclictest21181-21expr08:35:151
121519983,4cyclictest311rcuc/208:25:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional