You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-31 - 18:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Tue Mar 31, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2802099260,25cyclictest23557-21ssh11:19:111
2803099250,2cyclictest5787-21ssh12:37:483
2802599250,1cyclictest5862-21ssh12:05:262
28020992523,1cyclictest0-21swapper/109:58:261
2802099250,25cyclictest0-21swapper/111:41:321
2802099250,24cyclictest31922-21ssh09:20:141
2802099250,24cyclictest13771-21ssh09:33:541
28018992523,1cyclictest18965-21cron09:05:000
2801899250,1cyclictest0-21swapper/012:03:430
28030992422,1cyclictest0-21swapper/311:47:363
2803099240,24cyclictest21361-21diskmemload12:08:083
2803099240,24cyclictest21361-21diskmemload10:57:163
2803099240,23cyclictest0-21swapper/310:23:433
2803099240,23cyclictest0-21swapper/309:38:153
2803099240,1cyclictest21361-21diskmemload09:25:363
2803099240,1cyclictest0-21swapper/309:48:183
28025992424,0cyclictest0-21swapper/212:12:092
28025992422,0cyclictest0-21swapper/211:00:302
28025992420,4cyclictest24417-21rm11:20:152
2802599241,22cyclictest21361-21diskmemload09:56:122
2802599240,2cyclictest2863-21ssh11:31:232
2802599240,23cyclictest6543-21perf08:40:012
2802599240,23cyclictest21361-21diskmemload11:25:312
2802599240,23cyclictest11494-21unixbench_multi10:03:342
2802599240,23cyclictest0-21swapper/210:58:402
28020992424,0cyclictest0-21swapper/112:30:191
2802099240,24cyclictest0-21swapper/110:06:151
2802099240,1cyclictest21361-21diskmemload11:21:291
2802099240,0cyclictest0-21swapper/109:13:571
28018992423,1cyclictest0-21swapper/011:13:410
28018992422,2cyclictest18266-21ssh11:46:360
28018992422,1cyclictest27630-21ssh10:53:010
2801899240,23cyclictest0-21swapper/011:26:070
2801899240,0cyclictest21361-21diskmemload11:07:530
2801899240,0cyclictest0-21swapper/011:52:360
2803099230,23cyclictest21361-21diskmemload09:21:133
2803099230,23cyclictest0-21swapper/311:17:513
2803099230,23cyclictest0-21swapper/310:47:143
2803099230,23cyclictest0-21swapper/310:08:093
2803099230,23cyclictest0-21swapper/309:30:283
2803099230,22cyclictest31006-21sh11:27:403
2803099230,22cyclictest26937-21sh10:51:503
2803099230,22cyclictest12390-21ssh11:40:173
2803099230,22cyclictest10051-21sendmail07:40:003
2803099230,22cyclictest0-21swapper/312:29:593
2803099230,1cyclictest7287-21ssh09:59:513
2803099230,1cyclictest21361-21diskmemload09:16:223
2803099230,0cyclictest21361-21diskmemload12:03:453
2803099230,0cyclictest21361-21diskmemload09:42:273
2803099230,0cyclictest0-21swapper/311:52:033
2803099230,0cyclictest0-21swapper/311:10:523
2803099230,0cyclictest0-21swapper/309:50:223
28025992323,0cyclictest0-21swapper/212:39:432
28025992323,0cyclictest0-21swapper/211:35:532
28025992323,0cyclictest0-21swapper/210:20:262
28025992323,0cyclictest0-21swapper/210:06:402
28025992322,1cyclictest0-21swapper/211:16:182
28025992321,1cyclictest27377-21df_inode09:48:262
28025992321,1cyclictest21361-21diskmemload09:30:322
28025992321,1cyclictest1105-21ssh12:00:542
28025992320,2cyclictest30487-21sh10:54:502
28025992320,2cyclictest21361-21diskmemload10:11:022
28025992320,2cyclictest15058-21ssh09:35:382
2802599230,23cyclictest21361-21diskmemload09:51:042
2802599230,23cyclictest0-21swapper/211:56:092
2802599230,23cyclictest0-21swapper/210:46:132
2802599230,23cyclictest0-21swapper/209:26:252
2802599230,22cyclictest9431-21rm10:33:452
2802599230,22cyclictest28984-21sh12:28:262
2802599230,22cyclictest22987-21sh10:15:052
2802599230,22cyclictest22987-21sh10:15:052
2802599230,22cyclictest21361-21diskmemload10:36:062
2802599230,22cyclictest14631-21ssh11:10:512
2802599230,0cyclictest0-21swapper/211:49:222
28020992323,0cyclictest21361-21diskmemload12:24:401
28020992323,0cyclictest0-21swapper/111:54:331
28020992323,0cyclictest0-21swapper/111:32:131
28020992323,0cyclictest0-21swapper/110:12:271
28020992323,0cyclictest0-21swapper/109:47:491
28020992323,0cyclictest0-21swapper/109:27:131
28020992321,1cyclictest30670-21ssh11:27:021
28020992321,1cyclictest0-21swapper/111:14:411
28020992321,1cyclictest0-21swapper/110:50:021
28020992320,2cyclictest7733-21rm11:36:011
2802099230,23cyclictest0-21swapper/111:02:021
2802099230,23cyclictest0-21swapper/110:40:071
2802099230,22cyclictest3299-21sendmail-msp11:00:011
2802099230,22cyclictest28818-21sh10:21:141
2802099230,22cyclictest25481-21sh10:49:541
2802099230,22cyclictest1163-21ssh12:01:001
2802099230,22cyclictest11319-21ssh10:36:251
2802099230,22cyclictest10788-21sh12:10:071
2802099230,0cyclictest0-21swapper/112:16:191
2802099230,0cyclictest0-21swapper/109:44:431
28018992323,0cyclictest21361-21diskmemload09:42:030
28018992323,0cyclictest0-21swapper/012:13:340
28018992323,0cyclictest0-21swapper/010:20:510
28018992323,0cyclictest0-21swapper/010:11:510
28018992323,0cyclictest0-21swapper/009:10:510
28018992322,1cyclictest31705-21kworker/u8:011:59:050
28018992322,1cyclictest13169-21kworker/u8:010:18:190
28018992322,1cyclictest13169-21kworker/u8:010:18:190
28018992322,1cyclictest0-21swapper/012:27:380
2801899230,2cyclictest14864-21ssh10:39:180
2801899230,23cyclictest0-21swapper/012:35:380
2801899230,23cyclictest0-21swapper/009:32:400
2801899230,22cyclictest7566-21latency_hist07:35:000
2801899230,22cyclictest7321-21sh11:35:240
2801899230,22cyclictest31927-21ssh12:30:410
2801899230,22cyclictest31423-21sh10:55:510
2801899230,22cyclictest30358-21idleruntime-cro07:15:010
2801899230,22cyclictest19339-21diskmemload11:15:130
2801899230,22cyclictest0-21swapper/012:21:410
2801899230,22cyclictest0-21swapper/010:48:260
2801899230,22cyclictest0-21swapper/009:26:030
2801899230,21cyclictest0-21swapper/009:15:110
2801899230,1cyclictest16979-21ssh12:16:150
2801899230,1cyclictest16821-21apt-get09:38:250
2801899230,0cyclictest0-21swapper/012:05:430
2801899230,0cyclictest0-21swapper/010:01:030
28030992222,0cyclictest0-21swapper/312:21:403
28030992222,0cyclictest0-21swapper/311:57:253
28030992222,0cyclictest0-21swapper/311:02:013
28030992222,0cyclictest0-21swapper/310:42:123
28030992220,1cyclictest29347-21lockfile-create08:20:003
28030992220,1cyclictest0-21swapper/311:33:513
28030992220,1cyclictest0-21swapper/310:26:583
2803099220,21cyclictest8420-21apt-get11:05:013
2803099220,21cyclictest14303-21apt-get08:58:253
2803099220,21cyclictest11332-21latency_hist08:50:013
2803099220,21cyclictest10735-21sh10:35:243
2803099220,21cyclictest0-21swapper/312:15:023
2803099220,21cyclictest0-21swapper/310:10:033
2803099220,1cyclictest23170-21ssh09:12:373
2803099220,1cyclictest11481-21ssh12:11:173
2803099220,1cyclictest0-21swapper/312:31:183
2803099220,1cyclictest0-21swapper/310:15:133
2803099220,1cyclictest0-21swapper/310:15:133
2803099220,0cyclictest21361-21diskmemload10:00:533
2803099220,0cyclictest0-21swapper/311:22:333
28025992222,0cyclictest27241-21ssh09:15:512
28025992222,0cyclictest0-21swapper/212:21:052
28025992221,1cyclictest0-21swapper/208:43:222
28025992220,1cyclictest606-21dbus-daemon08:00:012
28025992220,1cyclictest1283-21sh10:25:592
28025992220,1cyclictest0-21swapper/212:15:022
28025992219,2cyclictest17502-21diskmemload10:43:092
2802599220,22cyclictest23710-21ssh09:43:352
2802599220,21cyclictest885-21apt-get07:23:252
2802599220,21cyclictest22986-21ssh11:50:592
2802599220,21cyclictest21625-21ssh09:10:022
2802599220,21cyclictest1784-21grep12:33:272
2802599220,21cyclictest0-21swapper/209:03:232
2802599220,1cyclictest32448-21ssh09:21:012
2802599220,1cyclictest26939-21grep08:15:002
2802599220,1cyclictest0-21swapper/208:25:002
2802599220,0cyclictest0-21swapper/211:40:032
28020992222,0cyclictest0-21swapper/112:37:521
28020992221,1cyclictest24012-21sh10:16:481
28020992221,1cyclictest24012-21sh10:16:481
28020992220,1cyclictest27018-21rm09:15:311
2802099220,22cyclictest0-21swapper/109:51:011
2802099220,21cyclictest27556-21sh11:55:071
2802099220,21cyclictest0-21swapper/110:30:251
2802099220,21cyclictest0-21swapper/107:23:251
2802099220,1cyclictest26768-21ssh12:25:361
2802099220,0cyclictest0-21swapper/109:39:391
28018992221,1cyclictest0-21swapper/009:20:140
28018992220,1cyclictest29365-21awk08:20:000
28018992220,1cyclictest22150-21munin-run08:05:010
28018992220,1cyclictest10076-21perf07:40:000
2801899220,22cyclictest0-21swapper/011:20:330
2801899220,21cyclictest934-21ssh10:25:280
2801899220,21cyclictest21361-21diskmemload09:55:290
2801899220,21cyclictest0-21swapper/010:06:530
2801899220,21cyclictest0-21swapper/009:45:420
2801899220,1cyclictest5775-21sh10:30:390
2801899220,1cyclictest0-21swapper/011:30:070
2801899220,0cyclictest0-21swapper/009:50:210
28030992120,1cyclictest7248-21ssh11:35:173
28030992119,1cyclictest8953-21munin-run08:45:013
2803099210,20cyclictest15449-21apt-get07:53:243
2803099210,0cyclictest0-21swapper/308:23:263
28025992119,1cyclictest14938-21perf07:50:012
2802599210,1cyclictest0-21swapper/209:08:252
28020992120,0cyclictest0-21swapper/112:05:481
2802099210,21cyclictest0-21swapper/111:45:071
2802099210,20cyclictest0-21swapper/108:58:251
2802099210,0cyclictest0-21swapper/110:25:091
2802099210,0cyclictest0-21swapper/108:48:241
2802099210,0cyclictest0-21swapper/107:38:241
2802099210,0cyclictest0-21swapper/107:33:251
28018992121,0cyclictest0-21swapper/008:55:010
28018992121,0cyclictest0-21swapper/008:13:250
28018992119,1cyclictest16889-21ssh10:42:050
28018992119,1cyclictest0-21swapper/008:00:010
2801899210,21cyclictest0-21swapper/011:40:030
2801899210,21cyclictest0-21swapper/011:00:180
28030992019,0cyclictest2093-21kworker/3:209:00:023
28025992020,0cyclictest30890-21apt-get07:18:242
28025992020,0cyclictest10604-21apt-get07:43:252
28025992020,0cyclictest0-21swapper/208:53:222
28025992020,0cyclictest0-21swapper/207:28:262
28025992018,1cyclictest16202-21awk09:00:012
2802599200,0cyclictest2253-21apt-get08:33:252
28020992020,0cyclictest0-21swapper/108:33:251
28020992018,1cyclictest7058-21apt-get08:43:221
28020992018,1cyclictest0-21swapper/107:15:011
28020992017,2cyclictest22133-21apt08:05:011
2802099200,0cyclictest0-21swapper/108:53:221
28018992020,0cyclictest0-21swapper/008:48:240
28018992020,0cyclictest0-21swapper/007:48:240
28018992019,1cyclictest28014-21cyclictest08:30:000
2801899200,20cyclictest0-21swapper/008:08:260
2801899200,0cyclictest0-21swapper/009:08:250
28025991919,0cyclictest0-21swapper/207:53:242
28025991919,0cyclictest0-21swapper/207:38:242
2802599190,19cyclictest0-21swapper/208:28:242
28018991919,0cyclictest0-21swapper/008:43:220
28030991817,1cyclictest3290-21apt-get07:28:263
2803099181,0cyclictest151rcu_preempt09:08:263
28025991818,0cyclictest0-21swapper/208:18:272
28025991818,0cyclictest0-21swapper/208:03:252
28025991817,1cyclictest9456-21apt-get08:48:242
28020991818,0cyclictest0-21swapper/108:18:271
28020991818,0cyclictest0-21swapper/107:28:261
28018991818,0cyclictest0-21swapper/008:58:250
28018991816,1cyclictest12504-21cron07:45:010
28030991717,0cyclictest0-21swapper/307:13:233
28030991716,1cyclictest21062-21irqstats08:03:293
2803099170,16cyclictest337-21awk07:20:013
28025991717,0cyclictest5688-21apt-get07:33:252
28018991717,0cyclictest0-21swapper/007:53:240
28018991714,2cyclictest606-21dbus-daemon08:35:010
28030991616,0cyclictest0-21swapper/308:33:253
28030991616,0cyclictest0-21swapper/308:13:253
2802599160,3cyclictest0-21swapper/207:13:232
28020991616,0cyclictest0-21swapper/108:13:251
28020991614,1cyclictest30915-21ntp_states08:23:311
28020991614,1cyclictest13051-21apt-get07:48:251
2801899160,15cyclictest0-21swapper/007:23:270
28030991515,0cyclictest0-21swapper/307:43:243
28030991515,0cyclictest0-21swapper/307:33:253
28030991515,0cyclictest0-21swapper/307:23:253
28020991513,1cyclictest6359-21/usr/sbin/munin08:38:361
28030991414,0cyclictest0-21swapper/307:45:143
2803099140,14cyclictest0-21swapper/308:50:433
2803099140,14cyclictest0-21swapper/308:35:253
2803099140,14cyclictest0-21swapper/308:25:153
2803099140,0cyclictest0-21swapper/308:06:463
2803099140,0cyclictest0-21swapper/307:55:213
2802599140,14cyclictest0-21swapper/208:06:332
28020991414,0cyclictest0-21swapper/109:00:481
28020991414,0cyclictest0-21swapper/107:50:111
2802099140,14cyclictest0-21swapper/108:25:221
2802099140,14cyclictest0-21swapper/108:06:331
2802099140,0cyclictest0-21swapper/107:40:071
28018991414,0cyclictest0-21swapper/007:16:430
2801899140,14cyclictest0-21swapper/008:35:200
2801899140,14cyclictest0-21swapper/007:25:240
2801899140,0cyclictest0-21swapper/008:21:030
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional