You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-31 - 18:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Sun May 31, 2026 12:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
122242176103,28sleep10-21swapper/107:08:391
121492175103,27sleep30-21swapper/307:07:413
123222169102,60sleep00-21swapper/007:09:550
120342161108,28sleep20-21swapper/207:06:122
3275221300,2sleep00-21swapper/010:45:190
241792750,2sleep10-21swapper/109:26:541
255702730,3chrt25569-21grep08:50:243
297442610,1sleep00-21swapper/011:30:320
62932520,1sleep20-21swapper/211:37:502
226272510,2sleep00-21swapper/012:38:160
248172500,1sleep024814-21ls10:15:210
128922500,9sleep334-21ksoftirqd/312:06:323
269072480,2sleep20-21swapper/209:53:082
138592470,4chrt0-21swapper/111:43:391
325842460,2sleep00-21swapper/009:57:020
268492440,2sleep10-21swapper/108:55:151
258682430,1chrt0-21swapper/009:52:050
9322420,2sleep20-21swapper/210:45:252
207932390,3sleep3331rcuc/310:12:233
135942370,1sleep00-21swapper/007:10:210
177742360,3chrt34-21ksoftirqd/311:22:013
28742350,2sleep10-21swapper/110:23:241
275212340,3sleep334-21ksoftirqd/310:41:033
234562340,15sleep122-21ksoftirqd/111:50:201
1260299331,29cyclictest8513-21ssh12:27:320
1260299331,29cyclictest19191-21ssh12:10:570
321682320,2chrt0-21swapper/311:56:493
1260599321,28cyclictest7908-21ssh12:26:573
1260599321,28cyclictest19866-21ssh10:35:253
1260399326,24cyclictest211rcuc/112:35:481
6742310,1sleep00-21swapper/010:21:110
285002310,2chrt0-21swapper/310:18:213
210212310,1sleep20-21swapper/211:00:222
1260599310,28cyclictest32655-21diskmemload12:11:553
1260499311,28cyclictest24402-21/usr/sbin/munin12:15:172
1260499310,29cyclictest13962-21ssh11:19:572
12603993124,5cyclictest32655-21diskmemload12:13:121
199562300,1sleep00-21swapper/009:47:550
1260599300,7cyclictest24284-21ssh11:50:383
1260299301,26cyclictest5472-21ssh12:00:470
36162290,1chrt0-21swapper/209:11:372
12605992923,5cyclictest52850irq/140-enp3s0-12:36:553
1260599291,25cyclictest13228-21ssh12:30:323
1260499291,26cyclictest32655-21diskmemload12:26:172
12602992924,3cyclictest17570-21ssh12:34:550
12602992923,4cyclictest8330-21ssh11:39:530
1260299291,4cyclictest28676-1kworker/u9:111:21:520
1260299291,3cyclictest1325-21df11:10:140
109022290,1sleep30-21swapper/309:17:093
326882280,2sleep20-21swapper/212:21:002
214652280,2sleep20-21swapper/211:49:342
12604992819,6cyclictest22341-21ssh11:25:222
1260499281,25cyclictest24051-21ssh12:39:402
1260499280,26cyclictest32655-21diskmemload11:50:512
1260399285,7cyclictest24713-21munin-run10:40:001
12603992825,2cyclictest0-21swapper/111:34:221
12603992824,2cyclictest211rcuc/112:33:541
12603992823,3cyclictest10008-21ssh11:16:001
1260399281,3cyclictest4520-21ssh12:25:041
1260399281,25cyclictest0-21swapper/112:01:411
12602992821,5cyclictest4104-21diskmemload12:24:520
1260299282,12cyclictest7444-21ssh09:15:110
1260299281,25cyclictest11820-21ssh11:41:370
1260299281,25cyclictest0-21swapper/011:29:030
158762270,2sleep20-21swapper/210:09:132
12605992723,2cyclictest0-21swapper/312:00:113
12605992723,2cyclictest0-21swapper/311:41:553
12605992722,3cyclictest5011-21ssh11:36:323
12604992723,2cyclictest0-21swapper/211:20:562
12604992722,3cyclictest20590-21ssh12:12:202
12604992722,3cyclictest1748-21ssh11:58:502
1260499271,3cyclictest1502-21perl11:10:152
1260499270,25cyclictest11809-21ssh11:41:352
1260399273,7cyclictest14381-21/usr/sbin/munin11:20:131
12603992724,2cyclictest211rcuc/111:59:101
12603992724,2cyclictest211rcuc/110:04:191
12603992721,4cyclictest22130-21tail11:25:181
1260399271,24cyclictest0-21swapper/112:15:071
12603992710,9cyclictest22-21ksoftirqd/111:49:351
1260299271,24cyclictest0-21swapper/011:51:030
1260299270,25cyclictest30985-21ssh11:55:380
1260299270,25cyclictest0-21swapper/011:15:480
12605992623,2cyclictest0-21swapper/311:48:433
12605992622,2cyclictest0-21swapper/312:24:043
1260599261,24cyclictest0-21swapper/311:17:013
1260499263,17cyclictest27722-21ssh09:30:102
12604992624,1cyclictest0-21swapper/212:00:362
1260499261,2cyclictest0-21swapper/211:05:252
1260399262,7cyclictest0-21swapper/110:10:071
12603992624,1cyclictest0-21swapper/110:47:531
12603992612,8cyclictest22-21ksoftirqd/109:54:241
1260399261,24cyclictest0-21swapper/111:35:231
1260399261,24cyclictest0-21swapper/111:14:271
1260399261,10cyclictest22-21ksoftirqd/110:29:071
1260399260,2cyclictest32486-21ssh12:20:471
1260299260,24cyclictest20039-21ssh11:48:060
205402250,2sleep30-21swapper/309:48:303
12605992522,2cyclictest32655-21diskmemload11:34:393
12605992522,2cyclictest0-21swapper/312:15:013
12605992522,2cyclictest0-21swapper/311:26:043
12605992513,8cyclictest2720-21sed09:35:173
1260599251,23cyclictest0-21swapper/311:13:293
12604992523,1cyclictest0-21swapper/212:30:122
1260499251,23cyclictest0-21swapper/212:08:422
1260499251,23cyclictest0-21swapper/211:34:422
1260399256,7cyclictest22-21ksoftirqd/109:34:081
1260399253,7cyclictest32655-21diskmemload12:06:531
1260399252,9cyclictest7789-21df09:15:131
1260399252,7cyclictest32157-21taskset07:51:021
1260399252,7cyclictest30992-21rm10:44:331
1260399251,8cyclictest18388-21ssh10:58:521
12603992514,7cyclictest0-21swapper/109:35:121
12603992513,7cyclictest12087-21fschecks_count10:30:151
1260299250,24cyclictest0-21swapper/012:15:380
1260599242,15cyclictest6279-21ssh10:50:043
12604992414,6cyclictest52950irq/141-enp3s0-10:50:122
1260399246,6cyclictest22-21ksoftirqd/107:41:031
1260399243,8cyclictest30534-21mailstats09:55:221
1260399243,8cyclictest22-21ksoftirqd/110:05:501
1260399242,8cyclictest32655-21diskmemload09:40:321
1260399242,8cyclictest22-21ksoftirqd/109:10:121
1260399242,7cyclictest22-21ksoftirqd/108:09:331
12603992420,2cyclictest211rcuc/110:54:071
12603992419,4cyclictest211rcuc/107:58:361
1260399241,7cyclictest22-21ksoftirqd/109:23:371
1260399241,15cyclictest0-21swapper/110:15:431
1260399240,4cyclictest0-21swapper/111:05:261
1260299246,7cyclictest141rcu_preempt10:44:470
1260299245,10cyclictest12-21ksoftirqd/010:31:330
12602992415,7cyclictest11158-21ntp_states12:05:240
12602992411,11cyclictest22187-21gzip09:25:220
12602992410,4cyclictest12-21ksoftirqd/009:40:290
224342230,3sleep228-21ksoftirqd/208:45:132
12605992315,6cyclictest15238-21ntp_states09:20:213
12605992313,7cyclictest560-21dbus-daemon07:50:003
1260499234,11cyclictest28-21ksoftirqd/210:38:252
1260399233,6cyclictest22-21ksoftirqd/108:00:171
1260399232,7cyclictest22-21ksoftirqd/108:43:541
1260399232,7cyclictest22-21ksoftirqd/108:10:071
1260399231,6cyclictest24653-21ssh11:03:231
1260399231,6cyclictest19873-21ssh09:47:501
12603992313,7cyclictest16874-21cron07:20:001
12603992312,8cyclictest24970-21grep07:35:261
12602992321,1cyclictest0-21swapper/010:28:010
12602992315,6cyclictest32108-21ssh09:33:020
12602992313,7cyclictest17487-21df_inode07:20:130
12602992310,2cyclictest141rcu_preempt09:12:300
117272230,11sleep012-21ksoftirqd/008:20:150
1260599222,14cyclictest692-21ssh09:10:133
12605992215,4cyclictest15299-21idleruntime-cro08:29:593
1260499221,5cyclictest0-21swapper/209:25:312
12604992212,7cyclictest278-21systemd-journal10:30:012
1260499221,19cyclictest15108-21rm10:55:372
1260399223,6cyclictest22-21ksoftirqd/108:26:091
1260399222,7cyclictest22-21ksoftirqd/107:48:531
12603992218,3cyclictest211rcuc/107:21:071
1260399220,5cyclictest0-21swapper/108:30:291
12602992216,4cyclictest19574-21cron08:39:590
12602992213,7cyclictest18793-21users10:10:230
1260599215,10cyclictest30645-21ssh10:20:133
1260599213,12cyclictest636-21runrttasks10:25:113
1260499211,4cyclictest0-21swapper/209:36:152
12604992113,6cyclictest3584-21apt10:00:122
12604992112,7cyclictest21435-21apt07:30:112
12604992112,6cyclictest31479-21smartctl07:50:222
1260399213,7cyclictest22-21ksoftirqd/108:20:201
1260399212,7cyclictest22-21ksoftirqd/108:51:061
12602992113,6cyclictest13232-21ssh10:06:340
12602992111,8cyclictest4944-21df_inode08:05:140
12602992111,7cyclictest14742-21cat10:55:250
1260599208,8cyclictest28628-21apt-get09:00:123
12605992013,5cyclictest30977-21cut07:50:183
12605992013,5cyclictest25569-21sendmail07:40:003
12605992012,6cyclictest6514-21basename08:10:013
12605992012,6cyclictest17667-21find08:35:103
12605992012,5cyclictest18950irq/143-ahci[0010:00:003
1260499209,7cyclictest52950irq/141-enp3s0-10:15:142
1260499209,3cyclictest321-21ssh09:57:142
12604992011,7cyclictest4667-21cat10:25:002
1260399202,4cyclictest22-21ksoftirqd/108:15:221
1260399201,7cyclictest32450-21runrttasks09:07:341
12603992017,2cyclictest211rcuc/108:37:001
1260399200,7cyclictest22-21ksoftirqd/107:32:161
1260299208,10cyclictest131rcuc/010:01:380
1260299206,3cyclictest141rcu_preempt10:52:000
1260299202,3cyclictest141rcu_preempt08:50:130
12602992012,6cyclictest3360-21ntpq08:00:210
12602992011,7cyclictest2416-21date08:00:000
12602992010,8cyclictest30363-21ssh11:07:220
1260299200,9cyclictest66050lldpd07:35:520
320302190,2sleep30-21swapper/310:45:143
1260599193,12cyclictest0-21swapper/310:55:113
1260599192,11cyclictest11679-21apt-get10:30:133
12605991912,5cyclictest580-21expr07:55:133
12605991912,5cyclictest4676-21awk10:00:213
12605991912,5cyclictest23287-21sed09:50:173
12605991912,5cyclictest21133-21cat09:25:123
12605991911,6cyclictest52850irq/140-enp3s0-09:30:093
12605991911,6cyclictest16480-21ntpq08:30:243
12605991911,5cyclictest27932-21cut11:05:233
12605991911,5cyclictest23476-21timerandwakeup08:45:243
1260499198,8cyclictest26618-21cut08:55:142
12604991913,4cyclictest29133-21sh10:42:412
12604991912,5cyclictest6934-21expr08:10:122
12604991912,5cyclictest582-21expr07:55:142
12604991912,5cyclictest4796-21expr08:05:132
12604991912,5cyclictest17510-21awk07:20:132
12604991912,4cyclictest8937-21timerwakeupswit09:15:242
12604991912,4cyclictest13760-21sh10:30:472
1260399192,6cyclictest22-21ksoftirqd/107:10:351
1260399192,5cyclictest22-21ksoftirqd/108:50:011
1260299195,3cyclictest141rcu_preempt09:22:400
1260299194,9cyclictest12-21ksoftirqd/010:38:580
1260299194,7cyclictest141rcu_preempt11:00:550
1260299192,2cyclictest141rcu_preempt09:35:190
12602991913,4cyclictest12-21ksoftirqd/007:20:010
12602991912,5cyclictest29199-21hostname09:00:160
12602991912,5cyclictest28015-21seq08:55:470
12602991912,5cyclictest16312-21memory08:30:210
12602991911,6cyclictest31458-21expr09:05:180
12602991910,7cyclictest27865-21seq07:44:200
1260599188,7cyclictest7471-21latency08:10:163
12605991811,5cyclictest16077-21gzip07:15:213
12605991811,5cyclictest12003-21cat08:20:183
12605991810,6cyclictest1-21systemd09:40:223
1260499188,7cyclictest0-21swapper/208:40:262
1260499182,11cyclictest14998-21apt-get07:15:132
12604991813,3cyclictest17771-21sh09:45:462
12604991813,3cyclictest11123-21sh09:40:542
12604991812,4cyclictest15122-21sh09:20:202
12604991811,5cyclictest3466-21awk08:00:232
12604991811,5cyclictest11846-21ls08:20:162
12604991810,6cyclictest28392-21expr07:45:152
12604991810,6cyclictest16924-21cat08:30:272
12604991810,6cyclictest13405-21cat08:25:122
12604991810,5cyclictest26416-21cut08:55:012
1260399182,4cyclictest0-21swapper/107:28:311
1260299188,2cyclictest12-21ksoftirqd/008:45:250
1260299183,11cyclictest30321-21apt-get07:50:110
12602991814,2cyclictest19298-21apt07:25:110
12602991813,3cyclictest28563-21fschecks_count07:45:170
12602991811,5cyclictest9094-21cat08:15:120
12602991811,5cyclictest6966-21expr08:10:120
12602991811,5cyclictest15051-21seq08:27:020
1260599173,8cyclictest9101-21apt-get08:15:133
12605991712,3cyclictest12619-21/usr/sbin/munin07:10:213
12605991711,4cyclictest11399-21ntp_kernel_pll_10:05:213
1260599171,14cyclictest0-21swapper/311:00:083
12605991710,5cyclictest30989-21expr09:05:133
12605991710,5cyclictest22334-21ntp_kernel_pll_07:30:193
12605991710,5cyclictest17547-21cut07:20:143
1260499179,6cyclictest29281-21kernelversion09:00:172
1260499179,6cyclictest24013-21perl07:35:162
1260499179,6cyclictest12630-21find07:10:112
1260499179,5cyclictest9164-21cat08:15:142
1260499176,8cyclictest18033-21cat08:35:152
1260499172,10cyclictest30766-21apt-get09:05:132
12604991712,3cyclictest16994-21sh10:10:062
1260499171,11cyclictest19299-21apt-get07:25:122
12604991710,5cyclictest26234-21expr07:40:142
12602991712,3cyclictest21746-21latency_hist08:45:010
12602991711,4cyclictest21625-21expr07:30:130
1260599169,5cyclictest26766-21latency07:40:183
1260599169,5cyclictest20210-21ntpq07:25:203
12605991612,2cyclictest20498-21irqstats08:40:183
1260599161,13cyclictest3440-21cut08:00:233
12605991611,3cyclictest27058-21idleruntime08:55:183
1260399169,5cyclictest28866-21expr09:00:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional