You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-02 - 22:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Sun Aug 02, 2026 12:45:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19386217196,26sleep30-21swapper/307:08:523
19472216496,61sleep00-21swapper/007:09:570
193352146110,25sleep10-21swapper/107:08:141
3090621430,2sleep10-21swapper/111:02:271
192442139105,23sleep20-21swapper/207:07:012
3028721010,2sleep00-21swapper/012:38:540
218722740,5sleep122-21ksoftirqd/109:20:171
35202700,2sleep10-21swapper/109:00:161
101062660,4sleep310099-21turbostat11:59:593
197522610,1sleep20-21swapper/211:42:172
86812570,4sleep10-21swapper/109:58:311
117112550,1sleep20-21swapper/209:36:162
253442520,1sleep30-21swapper/309:22:493
189132510,1sleep10-21swapper/112:30:151
165922490,1sleep10-21swapper/111:40:151
274762480,1sleep10-21swapper/110:35:261
115932440,1sleep20-21swapper/210:00:192
168652430,2sleep00-21swapper/010:52:020
100192410,2sleep00-21swapper/010:46:530
71232390,1sleep20-21swapper/207:55:002
169032380,1sleep30-21swapper/310:28:343
1993399361,7cyclictest12-21ksoftirqd/010:10:010
97052320,1chrt0-21swapper/210:46:352
19933993212,4cyclictest12-21ksoftirqd/011:09:500
1993399319,10cyclictest12-21ksoftirqd/011:24:410
1993399318,12cyclictest12-21ksoftirqd/009:25:230
19933993111,10cyclictest12-21ksoftirqd/009:44:440
19933993110,10cyclictest12-21ksoftirqd/012:26:250
19933993110,10cyclictest12-21ksoftirqd/010:16:480
19935993012,3cyclictest28-21ksoftirqd/211:56:442
19935993011,5cyclictest28-21ksoftirqd/210:26:522
19935993011,4cyclictest28-21ksoftirqd/212:03:492
19935993011,4cyclictest28-21ksoftirqd/212:03:492
1993399309,10cyclictest12-21ksoftirqd/010:59:210
19933993011,9cyclictest12-21ksoftirqd/007:47:540
1993599297,10cyclictest28-21ksoftirqd/209:33:152
1993599294,6cyclictest28-21ksoftirqd/209:40:152
19935992915,2cyclictest141rcu_preempt10:40:342
19935992912,5cyclictest28-21ksoftirqd/209:57:192
19935992910,5cyclictest28-21ksoftirqd/211:52:572
1993399299,8cyclictest12-21ksoftirqd/012:13:390
1993399299,7cyclictest141rcu_preempt11:01:520
1993399297,13cyclictest12-21ksoftirqd/012:16:100
19933992922,5cyclictest12-21ksoftirqd/009:20:010
33742280,3sleep10-21swapper/110:18:331
1993599289,10cyclictest28-21ksoftirqd/211:05:082
1993599283,15cyclictest28-21ksoftirqd/210:50:132
19935992814,4cyclictest28-21ksoftirqd/211:01:452
19935992812,4cyclictest28-21ksoftirqd/211:37:092
19935992810,10cyclictest28-21ksoftirqd/212:12:442
1993399288,9cyclictest12-21ksoftirqd/009:56:010
1993399286,14cyclictest12-21ksoftirqd/009:38:350
1993399284,5cyclictest12-21ksoftirqd/011:49:590
1993399280,7cyclictest10562-21/usr/sbin/munin11:35:210
19935992710,9cyclictest28-21ksoftirqd/212:38:292
1993399278,10cyclictest12-21ksoftirqd/010:26:350
1993399277,9cyclictest12-21ksoftirqd/010:42:060
1993399277,10cyclictest12-21ksoftirqd/011:42:020
1993399276,10cyclictest12-21ksoftirqd/009:49:380
1993399275,10cyclictest12-21ksoftirqd/012:08:120
1993399272,6cyclictest12-21ksoftirqd/010:30:060
1993399272,11cyclictest12-21ksoftirqd/010:38:400
1993399271,24cyclictest0-21swapper/007:15:310
19933992710,9cyclictest12-21ksoftirqd/011:56:480
19933992710,8cyclictest12-21ksoftirqd/011:10:520
1993399270,11cyclictest12-21ksoftirqd/010:00:250
1993999261,23cyclictest0-21swapper/307:17:563
1993599267,9cyclictest141rcu_preempt12:05:512
1993599267,6cyclictest28-21ksoftirqd/209:54:332
1993599265,11cyclictest28-21ksoftirqd/209:21:052
19935992613,4cyclictest28-21ksoftirqd/210:10:392
19935992611,5cyclictest28-21ksoftirqd/209:28:372
19935992610,4cyclictest28-21ksoftirqd/211:21:362
1993499260,25cyclictest0-21swapper/107:17:531
1993399269,8cyclictest12-21ksoftirqd/008:49:060
1993399269,6cyclictest12-21ksoftirqd/012:32:550
1993399268,6cyclictest12-21ksoftirqd/010:20:250
1993399267,9cyclictest12-21ksoftirqd/009:51:410
1993399265,9cyclictest12-21ksoftirqd/010:10:170
19933992624,1cyclictest0-21swapper/007:14:590
19933992617,6cyclictest12-21ksoftirqd/011:20:000
19933992615,6cyclictest636-21runrttasks09:00:130
19933992610,8cyclictest12-21ksoftirqd/011:26:530
137762260,1sleep10-21swapper/112:25:421
19939992514,8cyclictest20328-21df_inode07:10:143
1993599259,10cyclictest28-21ksoftirqd/210:36:222
1993599257,2cyclictest141rcu_preempt11:15:202
1993599257,11cyclictest28-21ksoftirqd/210:30:112
1993599256,4cyclictest28-21ksoftirqd/211:48:472
1993599255,11cyclictest28-21ksoftirqd/210:59:372
1993599252,16cyclictest11791-21ssh12:25:122
19935992518,5cyclictest28-21ksoftirqd/208:20:002
19935992518,5cyclictest28-21ksoftirqd/208:15:002
19935992510,5cyclictest6719-21sh10:20:242
19935992510,5cyclictest28-21ksoftirqd/212:16:202
1993499251,8cyclictest30887-21python309:50:271
19934992515,6cyclictest22-21ksoftirqd/107:40:101
1993399258,9cyclictest12-21ksoftirqd/007:53:140
1993399255,9cyclictest12-21ksoftirqd/009:31:100
1993399255,11cyclictest12-21ksoftirqd/012:02:450
1993399255,11cyclictest12-21ksoftirqd/012:02:440
19933992510,6cyclictest12-21ksoftirqd/012:24:080
85952240,2sleep30-21swapper/312:22:273
46922240,2sleep00-21swapper/011:30:350
19939992410,5cyclictest13208-21cut10:50:013
1993599249,5cyclictest28-21ksoftirqd/211:11:142
1993599249,4cyclictest28-21ksoftirqd/212:32:412
1993599243,5cyclictest28-21ksoftirqd/210:19:442
1993599243,3cyclictest28-21ksoftirqd/209:48:172
19935992422,1cyclictest0-21swapper/207:15:342
19935992412,4cyclictest28-21ksoftirqd/212:21:272
1993399249,3cyclictest12-21ksoftirqd/011:50:500
1993399244,8cyclictest12-21ksoftirqd/009:10:280
1993399241,8cyclictest12-21ksoftirqd/009:21:250
19933992414,7cyclictest11496-21cron08:05:000
206042230,1sleep30-21swapper/310:30:313
19939992314,7cyclictest3845-21ntpq11:30:203
19939992313,6cyclictest52950irq/141-enp3s0-12:30:133
1993599239,4cyclictest28-21ksoftirqd/208:05:172
1993599239,3cyclictest141rcu_preempt11:25:082
1993599238,4cyclictest28-21ksoftirqd/209:17:022
1993499238,13cyclictest6036-21ntpq07:50:221
1993499231,7cyclictest31147-21grep12:15:181
1993499231,7cyclictest1425-21/usr/sbin/munin11:05:231
1993399239,4cyclictest12-21ksoftirqd/008:43:420
1993399238,7cyclictest12-21ksoftirqd/008:17:000
1993399237,8cyclictest12-21ksoftirqd/007:24:360
1993399235,9cyclictest12-21ksoftirqd/007:28:020
1993399232,15cyclictest22489-21taskset08:30:130
32342220,4chrt3232-21memory09:30:193
31092220,3sleep23097-21ssh11:30:132
19939992216,4cyclictest34-21ksoftirqd/307:59:593
1993999221,5cyclictest0-21swapper/312:25:193
19939992212,7cyclictest9501-21df_inode09:35:143
1993599227,2cyclictest141rcu_preempt07:40:172
1993599222,15cyclictest19962-21apt-get07:10:032
19935992215,5cyclictest28-21ksoftirqd/208:35:002
19935992214,6cyclictest28-21ksoftirqd/207:30:002
19935992213,7cyclictest19455-21seq10:05:442
1993499222,17cyclictest15297-21grep11:15:161
19934992213,7cyclictest7192-21diskstats10:45:131
19934992213,6cyclictest13952-21forks10:50:171
1993399227,8cyclictest12-21ksoftirqd/007:34:130
1993399227,7cyclictest12-21ksoftirqd/008:56:220
1993399227,7cyclictest12-21ksoftirqd/008:56:210
1993399220,8cyclictest7165-21cron09:10:000
1993399220,6cyclictest12-21ksoftirqd/008:35:180
187042220,2sleep30-21swapper/309:17:593
158552220,2sleep00-21swapper/008:15:010
268602210,2sleep10-21swapper/111:23:581
1993999211,9cyclictest28070-21apt-key11:25:013
1993999211,5cyclictest10716-21tr12:00:143
1993999211,5cyclictest10716-21tr12:00:133
19939992115,4cyclictest34-21ksoftirqd/307:24:593
19939992115,4cyclictest23526-21sh12:10:003
19939992113,6cyclictest3928-21pluginstate07:45:213
19939992113,6cyclictest12328-21smartctl08:05:203
19939992112,7cyclictest25095-21awk12:10:223
19939992111,7cyclictest28139-21cat08:40:253
1993599216,6cyclictest11011-21sed08:00:312
1993599215,4cyclictest28-21ksoftirqd/209:12:062
1993599212,11cyclictest0-21swapper/208:59:192
1993599212,11cyclictest0-21swapper/208:59:182
19935992115,4cyclictest28-21ksoftirqd/208:00:002
19935992114,5cyclictest4872-21grep07:50:002
1993599211,13cyclictest0-21swapper/208:52:202
1993599211,11cyclictest28-21ksoftirqd/208:25:012
1993499212,12cyclictest24624-21apt-get08:35:131
19934992114,4cyclictest636-21runrttasks11:50:121
19934992113,6cyclictest21083-21memory10:55:191
19934992113,6cyclictest13075-21seq12:01:131
19934992113,6cyclictest13075-21seq12:01:131
19934992112,7cyclictest16628-21seq10:04:401
19934992112,7cyclictest13076-21ntpq10:25:201
19934992112,5cyclictest9075-21cron11:35:011
19934992111,7cyclictest6282-21cat12:20:251
1993399219,4cyclictest12-21ksoftirqd/008:08:000
19933992114,4cyclictest9148-21grep07:59:590
1993399210,8cyclictest32731-21sendmail07:40:010
65972201,1sleep10-21swapper/109:05:261
1993999208,4cyclictest141rcu_preempt09:42:323
1993999208,4cyclictest10200-21ssh10:23:373
1993999202,12cyclictest28563-21taskset11:25:133
1993999201,6cyclictest18665-21mailstats10:05:213
19939992014,3cyclictest1335-21sh09:29:273
19939992012,6cyclictest9968-21cat08:00:173
19939992012,6cyclictest11978-21sensors_temp10:00:243
19939992011,7cyclictest13347-21seq11:37:343
19939992011,6cyclictest5277-21cat09:05:123
19939992011,6cyclictest17611-21grep11:40:263
1993599205,4cyclictest28-21ksoftirqd/208:49:352
19935992014,4cyclictest28-21ksoftirqd/208:40:002
19934992013,5cyclictest31804-31updatedb.mlocat07:35:231
19934992012,6cyclictest32765-21ls10:40:131
19934992012,6cyclictest30590-21wc10:15:001
19934992012,6cyclictest29227-21latency_hist11:50:001
19934992012,6cyclictest25927-21seq07:21:471
19934992012,6cyclictest11525-21seq11:12:051
19934992012,5cyclictest10346-21ls08:00:221
1993399202,4cyclictest0-21swapper/008:20:220
19933992012,5cyclictest2035-21grep07:40:240
1993999192,12cyclictest22740-21sh09:45:123
19939991914,3cyclictest25963-21sh10:59:283
19939991913,4cyclictest28158-21sh12:36:443
19939991913,4cyclictest16069-21uname11:15:233
19939991913,4cyclictest13347-21sh11:13:543
19939991912,5cyclictest28056-21seq10:12:343
19939991912,5cyclictest28019-21sed11:00:193
19939991911,6cyclictest6041-21uname07:50:233
19939991911,6cyclictest5639-21seq09:55:283
19939991911,6cyclictest5477-21ssh10:44:053
19939991911,6cyclictest30971-21df12:15:163
19939991911,6cyclictest14437-21wc10:50:203
19939991910,6cyclictest22893-21apt-get11:45:113
1993999190,6cyclictest7174-21diskmemload10:15:263
1993599196,4cyclictest28-21ksoftirqd/209:00:242
19935991914,4cyclictest28-21ksoftirqd/207:40:002
19935991913,4cyclictest28-21ksoftirqd/208:30:012
1993499196,10cyclictest0-21swapper/110:20:051
1993499192,11cyclictest986-21apt-get08:55:141
1993499192,11cyclictest986-21apt-get08:55:141
19934991914,3cyclictest10480-21sh09:11:331
19934991913,4cyclictest7070-21sh09:33:221
19934991913,4cyclictest20716-21sh10:30:381
19934991913,3cyclictest25877-21cut12:35:161
19934991912,4cyclictest7174-21diskmemload09:43:011
19934991911,6cyclictest5410-21cat11:55:251
19934991911,6cyclictest28544-21seq08:43:381
19934991911,6cyclictest22951-21ls09:45:141
19934991911,6cyclictest18903-21ls10:05:241
19934991911,5cyclictest31337-21cat08:50:131
19934991911,5cyclictest27414-21cut07:25:211
19934991911,5cyclictest22959-21expr08:30:171
19934991910,7cyclictest29735-21python307:30:211
19934991910,6cyclictest22-21ksoftirqd/107:45:121
19934991910,6cyclictest14334-21perl08:10:151
1993399190,7cyclictest12-21ksoftirqd/008:25:190
1993399190,6cyclictest12-21ksoftirqd/008:50:190
1993999189,6cyclictest29882-21memory08:45:163
1993999189,6cyclictest15111-21cat08:10:233
1993999187,8cyclictest1321-21diskstats08:55:173
1993999187,8cyclictest1321-21diskstats08:55:163
19939991813,3cyclictest9846-21sh09:10:563
19939991813,3cyclictest886-21sh11:52:293
19939991813,3cyclictest4218-21sh11:06:283
19939991811,5cyclictest25182-21smartctl08:35:173
19939991810,6cyclictest30236-21cat09:50:213
19939991810,4cyclictest7174-21diskmemload10:35:033
1993999180,6cyclictest18635-21fschecks_time08:20:183
1993599189,2cyclictest141rcu_preempt09:09:542
1993599182,3cyclictest141rcu_preempt08:40:202
1993599181,7cyclictest0-21swapper/207:33:332
1993499182,12cyclictest27965-21apt-get09:25:131
1993499182,11cyclictest16038-21apt-get08:15:121
1993499182,10cyclictest14369-21apt-get09:15:101
1993499181,7cyclictest21256-21ssh12:07:431
19934991813,3cyclictest1523-21sh11:29:101
19934991812,4cyclictest25014-21ntp_offset12:10:221
19934991812,4cyclictest12824-21sh11:37:021
19934991811,5cyclictest21535-21seq07:10:401
19934991811,5cyclictest12756-21awk08:05:271
19934991810,6cyclictest9838-21cat09:35:171
19934991810,6cyclictest29326-21cat08:45:121
19934991810,6cyclictest21758-21grep08:25:271
1993999179,6cyclictest4771-21seq09:01:463
1993999172,10cyclictest30718-21apt-get07:35:133
1993999171,11cyclictest28580-21apt-get07:30:113
19939991710,5cyclictest26821-21grep07:25:153
19939991710,5cyclictest24555-21ls08:35:013
19939991710,5cyclictest20517-21cat08:25:143
19935991712,3cyclictest24281-21/usr/sbin/munin07:20:172
19934991712,3cyclictest18176-21/usr/sbin/munin08:20:131
19934991710,5cyclictest8095-21expr07:55:231
1993999168,6cyclictest1471-21ntpq07:40:193
1993999161,11cyclictest31264-21apt-get08:50:123
1993999148,4cyclictest0-21swapper/308:15:123
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional