You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-06 - 02:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Tue May 05, 2026 12:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1733299331,30cyclictest313-21systemd-journal10:25:013
1733299280,24cyclictest32075-21ssh10:00:323
1733299270,26cyclictest0-21swapper/312:16:283
17332992621,4cyclictest11078-21diskmemload11:23:433
1733299260,25cyclictest17663-21ssh11:42:373
17332992524,1cyclictest5123-21ssh10:05:183
17332992521,3cyclictest11078-21diskmemload12:23:583
17332992520,4cyclictest12809-21ntp_states10:41:263
1733299250,1cyclictest0-21swapper/310:34:303
17327992523,1cyclictest0-21swapper/211:41:012
17327992523,1cyclictest0-21swapper/210:19:332
17327992522,2cyclictest0-21swapper/210:29:142
1732799250,24cyclictest32352-21/usr/sbin/munin10:31:232
1732799250,24cyclictest22686-21ssh11:46:412
1732799250,21cyclictest0-21swapper/209:15:072
1732799250,1cyclictest0-21swapper/210:24:592
1732399250,24cyclictest0-21swapper/109:23:211
1732399250,1cyclictest0-21swapper/112:26:551
1732399250,0cyclictest0-21swapper/112:02:071
17319992523,1cyclictest28845-21awk09:25:010
1731999250,24cyclictest11078-21diskmemload12:12:160
17332992422,2cyclictest0-21swapper/309:32:533
17332992420,3cyclictest11078-21diskmemload12:09:503
17332992420,3cyclictest0-21swapper/309:16:183
1733299240,24cyclictest0-21swapper/309:51:323
1733299240,23cyclictest606-21dbus-daemon07:35:003
1733299240,21cyclictest11078-21diskmemload11:53:353
1733299240,21cyclictest0-21swapper/309:22:043
1733299240,1cyclictest28896-21apt-get11:25:033
1733299240,0cyclictest11078-21diskmemload12:28:523
1733299240,0cyclictest11078-21diskmemload11:30:123
1733299240,0cyclictest11078-21diskmemload10:57:063
1733299240,0cyclictest11078-21diskmemload10:16:053
1733299240,0cyclictest0-21swapper/311:49:163
17327992420,4cyclictest0-21swapper/212:08:082
1732799240,24cyclictest0-21swapper/212:25:172
1732799240,23cyclictest11078-21diskmemload11:26:452
1732799240,1cyclictest30690-21cron11:55:012
17323992424,0cyclictest0-21swapper/109:39:091
17323992422,1cyclictest5500-21ssh12:30:091
1732399240,24cyclictest11078-21diskmemload10:14:381
1732399240,24cyclictest0-21swapper/111:39:151
1732399240,24cyclictest0-21swapper/110:33:221
1732399240,23cyclictest6610-21sh11:32:581
1732399240,23cyclictest16550-21perf12:40:011
1732399240,23cyclictest0-21swapper/109:45:531
1732399240,1cyclictest0-21swapper/109:16:521
1732399240,0cyclictest0-21swapper/110:22:201
1732399240,0cyclictest0-21swapper/109:53:231
17319992424,0cyclictest0-21swapper/011:58:230
17319992423,1cyclictest21701-21ssh12:16:060
17319992422,2cyclictest0-21swapper/011:37:490
17319992421,2cyclictest0-21swapper/010:46:260
1731999240,24cyclictest11078-21diskmemload09:57:320
1731999240,24cyclictest0-21swapper/010:50:560
1731999240,23cyclictest6288-21ssh12:31:100
1731999240,23cyclictest26888-21ssh11:22:120
1731999240,23cyclictest11078-21diskmemload12:38:430
1731999240,23cyclictest11078-21diskmemload09:18:390
1731999240,23cyclictest0-21swapper/010:20:140
1731999240,0cyclictest7166-21ssh10:08:100
17332992323,0cyclictest11078-21diskmemload10:11:443
17332992323,0cyclictest0-21swapper/312:02:543
17332992323,0cyclictest0-21swapper/311:08:133
17332992323,0cyclictest0-21swapper/309:10:333
17332992320,3cyclictest0-21swapper/309:35:033
17332992318,4cyclictest0-21swapper/312:13:083
1733299230,23cyclictest0-21swapper/310:36:573
1733299230,23cyclictest0-21swapper/309:59:253
1733299230,22cyclictest9718-21sh12:33:343
1733299230,22cyclictest4444-21ssh11:02:243
1733299230,22cyclictest313-21systemd-journal09:10:003
1733299230,22cyclictest31116-21sh11:55:153
1733299230,22cyclictest25952-21ssh10:52:563
1733299230,22cyclictest18414-21ssh11:15:533
1733299230,20cyclictest0-21swapper/309:48:273
1733299230,0cyclictest0-21swapper/310:26:363
17327992323,0cyclictest0-21swapper/212:35:002
17327992323,0cyclictest0-21swapper/212:15:452
17327992323,0cyclictest0-21swapper/211:07:002
17327992323,0cyclictest0-21swapper/209:37:262
17327992321,1cyclictest0-21swapper/211:15:582
17327992321,1cyclictest0-21swapper/210:53:262
17327992320,2cyclictest0-21swapper/209:55:192
17327992311,4cyclictest0-21swapper/209:28:372
1732799230,2cyclictest0-21swapper/211:02:232
1732799230,23cyclictest11078-21diskmemload10:48:572
1732799230,23cyclictest0-21swapper/211:55:182
1732799230,23cyclictest0-21swapper/211:31:072
1732799230,22cyclictest11729-21sh12:35:552
1732799230,22cyclictest11078-21diskmemload10:02:312
1732799230,1cyclictest31872-21apt-get08:46:212
1732799230,0cyclictest0-21swapper/212:00:362
1732799230,0cyclictest0-21swapper/210:42:022
17323992323,0cyclictest0-21swapper/109:56:521
17323992321,1cyclictest31256-21ssh11:55:261
17323992321,1cyclictest0-21swapper/111:40:401
17323992321,1cyclictest0-21swapper/110:36:231
17323992321,1cyclictest0-21swapper/110:27:361
1732399230,23cyclictest0-21swapper/110:02:161
1732399230,22cyclictest29706-21sh11:25:141
1732399230,22cyclictest11081-21cron09:10:001
1732399230,1cyclictest23226-21sh11:19:571
1732399230,1cyclictest0-21swapper/110:55:571
1732399230,0cyclictest0-21swapper/109:11:121
17319992323,0cyclictest11078-21diskmemload10:32:220
17319992323,0cyclictest0-21swapper/012:23:420
17319992323,0cyclictest0-21swapper/009:53:200
17319992322,0cyclictest11078-21diskmemload10:02:050
1731999232,0cyclictest62550irq/127-eno111:19:310
17319992320,2cyclictest4816-21sh10:35:110
1731999230,2cyclictest20681-21basename11:46:210
1731999230,23cyclictest0-21swapper/011:13:310
1731999230,23cyclictest0-21swapper/009:46:160
1731999230,22cyclictest28725-21sh11:52:150
1731999230,22cyclictest1460-21cron08:50:010
1731999230,22cyclictest13164-21ssh12:07:310
1731999230,22cyclictest1013-21ssh09:28:340
1731999230,0cyclictest11078-21diskmemload09:11:100
1731999230,0cyclictest0-21swapper/012:02:220
1731999230,0cyclictest0-21swapper/011:25:060
1731999230,0cyclictest0-21swapper/008:56:210
17332992220,2cyclictest0-21swapper/310:49:453
17332992220,2cyclictest0-21swapper/309:40:433
1733299220,21cyclictest0-21swapper/312:35:553
1733299220,1cyclictest29471-1kworker/u9:011:11:163
1733299220,1cyclictest24547-21latency_hist07:25:003
1733299220,1cyclictest0-21swapper/311:39:273
1733299220,1cyclictest0-21swapper/307:26:283
17327992222,0cyclictest11078-21diskmemload09:30:082
17327992222,0cyclictest0-21swapper/211:35:522
17327992222,0cyclictest0-21swapper/209:41:032
17327992222,0cyclictest0-21swapper/209:21:092
17327992220,1cyclictest7907-21ssh10:37:012
17327992220,1cyclictest12260-21ssh10:12:202
17327992220,1cyclictest11889-21ssh09:10:452
17327992220,1cyclictest0-21swapper/209:50:032
1732799220,22cyclictest0-21swapper/211:20:522
1732799220,22cyclictest0-21swapper/210:56:262
1732799220,21cyclictest0-21swapper/212:10:472
1732799220,21cyclictest0-21swapper/208:16:212
1732799220,21cyclictest0-21swapper/207:50:012
1732799220,1cyclictest0-21swapper/211:11:412
1732799220,1cyclictest0-21swapper/210:06:462
17323992221,1cyclictest2108-21perf09:30:011
17323992220,2cyclictest11078-21diskmemload11:00:051
17323992220,1cyclictest24541-21awk07:25:001
17323992220,1cyclictest19467-21munin-run08:20:001
17323992220,1cyclictest14244-21turbostat08:10:011
17323992220,1cyclictest0-21swapper/111:20:571
17323992219,2cyclictest12977-21sh11:11:041
1732399220,21cyclictest24912-21apt-get07:26:211
1732399220,21cyclictest11078-21diskmemload09:40:241
1732399220,21cyclictest10086-21apt-get10:40:031
1732399220,21cyclictest0-21swapper/112:20:081
1732399220,21cyclictest0-21swapper/112:05:071
1732399220,21cyclictest0-21swapper/111:50:201
1732399220,21cyclictest0-21swapper/110:15:141
1732399220,21cyclictest0-21swapper/110:05:011
1732399220,1cyclictest32625-21apt-get07:41:211
1732399220,1cyclictest25910-21sh12:19:191
1732399220,1cyclictest16174-21ssh12:11:161
1732399220,1cyclictest0-21swapper/111:05:091
1732399220,1cyclictest0-21swapper/110:45:111
1732399220,0cyclictest0-21swapper/109:33:261
17319992222,0cyclictest11078-21diskmemload11:30:240
17319992220,1cyclictest12019-21ssh09:40:130
1731999220,2cyclictest17907-21ssh11:42:560
1731999220,22cyclictest0-21swapper/010:17:010
1731999220,21cyclictest26571-21ssh10:26:040
1731999220,21cyclictest11078-21diskmemload10:40:150
1731999220,21cyclictest0-21swapper/009:35:320
1731999220,21cyclictest0-21swapper/009:30:050
1731999220,1cyclictest0-21swapper/011:00:200
1731999220,1cyclictest0-21swapper/010:56:330
17332992119,1cyclictest26698-21latency_hist08:35:003
1733299210,1cyclictest11845-21sleep08:05:013
1733299210,0cyclictest0-21swapper/307:51:233
17327992121,0cyclictest0-21swapper/209:45:382
17327992120,0cyclictest29477-21apt-get08:41:202
1732799210,20cyclictest9816-21apt-get08:01:222
1732799210,20cyclictest0-21swapper/208:06:212
1732799210,20cyclictest0-21swapper/207:11:212
1732799210,1cyclictest32624-21apt07:41:212
1732799210,1cyclictest11078-21diskmemload12:20:252
17323992120,1cyclictest1857-21apt-get08:51:221
17323992120,1cyclictest0-21swapper/109:01:221
17323992119,1cyclictest29349-21apt-get07:35:021
1732399210,21cyclictest0-21swapper/111:45:171
1732399210,1cyclictest9470-21idleruntime-cro08:00:011
1732399210,0cyclictest0-21swapper/108:26:211
17319992119,1cyclictest622-21gdbus08:05:010
1731999210,0cyclictest27061-21apt-get08:36:220
1731999210,0cyclictest24662-21apt-get08:31:190
1731999210,0cyclictest11078-21diskmemload12:25:190
1731999210,0cyclictest0-21swapper/008:26:210
1731999210,0cyclictest0-21swapper/008:06:220
1733299200,20cyclictest0-21swapper/307:20:013
1733299200,0cyclictest0-21swapper/307:11:223
17327992020,0cyclictest27310-21apt-get07:31:212
17327992020,0cyclictest0-21swapper/207:35:022
17323992020,0cyclictest20101-21apt-get07:16:211
17323992020,0cyclictest0-21swapper/108:21:221
17323992020,0cyclictest0-21swapper/108:10:021
17323992020,0cyclictest0-21swapper/107:31:211
17323992019,1cyclictest5006-21apt-get07:51:231
1732399200,20cyclictest0-21swapper/108:56:211
17319992019,1cyclictest19699-21awk07:15:010
1731999200,19cyclictest0-21swapper/007:21:220
1731999200,0cyclictest17402-21apt-get08:16:210
17332991919,0cyclictest22267-21apt-get08:26:213
17332991919,0cyclictest0-21swapper/307:36:203
17332991917,1cyclictest19470-21perf08:20:003
17327991919,0cyclictest0-21swapper/208:36:222
1732399190,18cyclictest0-21swapper/107:15:011
1732399190,0cyclictest0-21swapper/108:41:201
17319991919,0cyclictest0-21swapper/008:11:210
17332991818,0cyclictest0-21swapper/308:25:003
17332991817,0cyclictest0-21swapper/308:41:213
17327991818,0cyclictest0-21swapper/208:51:212
17327991818,0cyclictest0-21swapper/208:31:192
17327991818,0cyclictest0-21swapper/207:26:212
17327991817,0cyclictest0-21swapper/209:06:212
17327991817,0cyclictest0-21swapper/208:26:212
17323991818,0cyclictest0-21swapper/108:46:211
17323991818,0cyclictest0-21swapper/108:01:221
17319991818,0cyclictest8694-21latency_hist09:05:000
17319991818,0cyclictest0-21swapper/008:41:210
17319991818,0cyclictest0-21swapper/007:36:200
17319991817,0cyclictest0-21swapper/007:26:210
1733299173,13cyclictest1552-21sendmail_mailtr07:41:293
1733299172,1cyclictest32615-21if_enp1s0.808:46:243
17332991715,1cyclictest14246-21apt-get08:10:023
17327991717,0cyclictest0-21swapper/207:16:212
17319991717,0cyclictest2264-21perf07:45:000
17319991717,0cyclictest0-21swapper/007:51:240
17332991616,0cyclictest12210-21apt-get08:06:213
1733299160,1cyclictest27479-21df_inode08:36:233
1733299160,0cyclictest0-21swapper/307:57:023
17327991614,1cyclictest0-21swapper/207:57:112
17327991614,1cyclictest0-21swapper/207:52:352
17323991616,0cyclictest0-21swapper/107:46:211
17319991616,0cyclictest0-21swapper/007:31:210
17332991515,0cyclictest4253-21apt-get08:56:213
17332991512,1cyclictest0-21swapper/308:51:213
1733299150,1cyclictest8717-21munin-run09:05:003
1733299150,15cyclictest0-21swapper/307:49:203
17327991513,1cyclictest6297-21sleep09:00:002
17323991515,0cyclictest0-21swapper/108:36:221
17319991515,0cyclictest0-21swapper/007:16:210
1731999150,1cyclictest3301-21sed08:51:300
1732799140,14cyclictest0-21swapper/209:00:332
1732799140,14cyclictest0-21swapper/208:21:072
1732799140,14cyclictest0-21swapper/208:10:582
1732799140,14cyclictest0-21swapper/207:20:142
1732399140,14cyclictest0-21swapper/108:32:081
17319991414,0cyclictest0-21swapper/009:05:110
17319991414,0cyclictest0-21swapper/007:55:140
1731999140,14cyclictest0-21swapper/007:45:070
1731999140,0cyclictest0-21swapper/008:20:270
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional