You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-21 - 05:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackaslot8s.osadl.org (updated Fri Aug 21, 2026 00:45:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
200282255207,39sleep20-21swapper/219:06:362
202152144108,25sleep10-21swapper/119:08:591
201722141103,26sleep00-21swapper/019:08:260
201602140103,26sleep30-21swapper/319:08:173
264632750,3chrt26434-21perf23:00:001
155432670,2sleep10-21swapper/121:39:501
153082610,2sleep30-21swapper/322:26:513
14322600,2sleep30-21swapper/322:40:243
190972570,3sleep119103-21cpuspeed_turbos00:30:131
118932560,1sleep30-21swapper/323:12:423
52242530,1sleep30-21swapper/300:20:083
36792530,2sleep10-21swapper/122:42:261
191172520,2sleep10-21swapper/122:54:261
191172520,2sleep10-21swapper/122:54:261
323852460,1sleep20-21swapper/223:51:462
2056799462,31cyclictest25589-21apt-get22:35:121
2056799450,1cyclictest0-21swapper/123:18:251
49832440,2chrt0-21swapper/323:31:163
20567994411,19cyclictest823-21seq19:37:471
20567994411,15cyclictest16989-21sh22:28:331
308872430,3chrt0-21swapper/022:15:010
2056799436,23cyclictest24717-21seq20:33:121
2056799436,23cyclictest24717-21seq20:33:121
2056799434,21cyclictest13008-21seq00:01:071
2056799421,25cyclictest24039-21ssh00:10:111
2056799421,23cyclictest31429-21ssh00:39:261
148372420,1sleep20-21swapper/223:39:262
2056799417,17cyclictest5877-21sh23:08:261
2056799411,31cyclictest32574-21sh23:04:261
2056799411,23cyclictest10973-21sh00:24:231
2056799409,17cyclictest636-21runrttasks19:55:481
2056799407,19cyclictest636-21runrttasks19:12:351
2056799405,20cyclictest636-21runrttasks20:17:471
2056799404,19cyclictest10109-21sh23:10:551
2056799401,24cyclictest640-21snmpd23:46:331
2056799401,23cyclictest31620-21sh21:26:561
2056799401,22cyclictest31367-21/usr/sbin/munin00:15:181
2056799401,21cyclictest640-21snmpd21:22:431
2056799401,21cyclictest640-21snmpd21:22:431
2056799393,20cyclictest11205-21sh23:35:451
2056799393,18cyclictest2392-21apt-get23:30:121
2056799392,22cyclictest20469-21chrt20:23:421
2056799391,24cyclictest22442-21/usr/sbin/munin23:20:211
2056799391,22cyclictest29323-21chrt20:44:281
2056799391,21cyclictest4737-21timerwakeupswit19:45:231
2056799382,19cyclictest29405-21grep23:50:141
2056799381,23cyclictest5496-21chrt21:04:431
2056799381,23cyclictest24537-21chrt19:18:501
2056799381,23cyclictest22934-21sh22:32:461
2056799381,22cyclictest4442-21users21:30:271
2056799381,20cyclictest32397-21sh21:51:271
2056799371,22cyclictest21235-21sh23:44:041
2056799371,22cyclictest18145-21fschecks_time22:05:161
2056799371,21cyclictest29844-21ssh23:25:271
2056799371,20cyclictest8405-21sh22:21:411
2056799371,20cyclictest30197-21sh22:14:311
2056799370,22cyclictest9180-21sh22:46:131
2056799370,21cyclictest7567-21sh23:57:381
2056799365,19cyclictest636-21runrttasks20:49:571
2056799361,22cyclictest11763-21seq20:04:461
2056799361,21cyclictest905-21seq20:50:571
2056799361,21cyclictest7672-21diskmemload21:43:011
2056799361,21cyclictest18671-21python300:05:251
2056799361,20cyclictest9380-21ssh21:58:531
2056799351,22cyclictest636-21runrttasks20:14:261
2056799351,20cyclictest24938-21sh21:45:441
2056799341,21cyclictest26469-21seq19:20:531
2056799341,19cyclictest11945-21sh00:25:071
2056799331,20cyclictest13937-21sh21:14:311
280662320,2sleep00-21swapper/021:25:000
280662320,2sleep00-21swapper/021:25:000
2056799322,25cyclictest0-21swapper/122:15:111
2056799321,16cyclictest0-21swapper/121:16:551
20567993211,18cyclictest12313-21expr20:05:141
2056799309,18cyclictest1892-21grep20:55:151
2056799302,23cyclictest0-21swapper/119:25:111
2056799300,3cyclictest15527-21ssh22:03:181
2056799298,18cyclictest6787-21which21:05:221
79952280,3sleep28000-21cpuspeed_turbos23:10:122
2056799287,18cyclictest29304-21cut19:30:121
2056799281,24cyclictest26102-21sort20:35:181
2056799280,1cyclictest0-21swapper/119:40:221
2056799280,1cyclictest0-21swapper/119:40:221
20568992713,8cyclictest87742sleep219:55:232
2056799272,22cyclictest5735-21cstates19:50:131
2056799272,21cyclictest0-21swapper/120:25:101
20569992619,5cyclictest20920-21ssh23:20:003
48532250,1sleep20-21swapper/223:07:242
2056999251,1cyclictest0-21swapper/319:14:423
2056999250,12cyclictest285682gunzip20:40:223
2056899251,1cyclictest0-21swapper/219:14:302
20566992523,1cyclictest0-21swapper/019:16:050
20566992523,1cyclictest0-21swapper/019:14:240
2056999243,8cyclictest4032-21apt-get21:55:033
2056999243,15cyclictest6682-21ssh22:45:133
2056999242,15cyclictest7672-21diskmemload21:35:133
20568992422,1cyclictest0-21swapper/219:15:562
2056699243,15cyclictest7672-21diskmemload00:00:040
20566992414,6cyclictest31227-21tr22:15:120
20566992412,3cyclictest58192expr19:50:130
324002230,2sleep00-21swapper/020:50:170
20569992313,7cyclictest278-21systemd-journal00:05:003
20569992311,8cyclictest25719-21apt-get00:35:133
20568992314,7cyclictest278-21systemd-journal20:15:002
20568992311,3cyclictest32383-21grep19:35:172
20566992312,7cyclictest52750irq/139-enp3s0-22:35:130
20566992311,8cyclictest8884-21apt-config19:55:220
2056699231,10cyclictest153092sh22:26:510
6872220,2sleep00-21swapper/023:28:440
241262220,2sleep00-21swapper/020:30:230
241262220,2sleep00-21swapper/020:30:230
20569992215,2cyclictest34-21ksoftirqd/300:27:253
20569992213,7cyclictest4045-21tune2fs21:00:153
20569992213,6cyclictest278-21systemd-journal20:20:003
20569992212,7cyclictest30958-21cat20:45:253
2056999221,11cyclictest34-21ksoftirqd/322:00:263
2056899222,8cyclictest6236-21munin-run22:45:002
20568992214,6cyclictest12004-21ntp_states22:00:222
20568992213,7cyclictest24204-21ls23:45:282
2056699221,3cyclictest131962ssh22:00:570
20566992211,9cyclictest636-21runrttasks22:55:080
2056699221,10cyclictest31998-21munin-run00:39:590
185602220,2sleep00-21swapper/022:53:530
185602220,2sleep00-21swapper/022:53:530
2056999219,3cyclictest34-21ksoftirqd/300:30:113
2056999216,13cyclictest34-21ksoftirqd/322:55:253
20569992119,1cyclictest0-21swapper/319:15:493
20569992116,3cyclictest621-21rm21:28:223
20569992113,6cyclictest15669-21sh23:40:013
20569992112,6cyclictest278-21systemd-journal20:35:033
20569992111,8cyclictest22227-21rm21:44:463
2056899212,13cyclictest3560-21apt-get21:55:032
2056899212,11cyclictest22909-21ssh21:45:122
2056899211,8cyclictest16174-21ssh21:40:122
20568992113,6cyclictest31390-21seq00:39:232
20568992113,6cyclictest12386-21diskstats20:05:142
20568992112,7cyclictest31173-21diskstats00:15:162
20568992112,7cyclictest24915-21perl22:10:162
20568992111,8cyclictest15906-21ntp_offset21:15:212
2056699213,8cyclictest29827-21tr21:50:140
2056699213,13cyclictest4659-21apt-get00:20:030
2056699212,7cyclictest1166-21ssh23:05:120
20566992114,5cyclictest12-21ksoftirqd/000:15:000
20566992113,6cyclictest31917-21cut00:15:240
20566992112,7cyclictest31029-21seq19:34:290
20566992112,6cyclictest278-21systemd-journal22:10:000
20566992112,6cyclictest21668-21cat23:20:130
20566992112,6cyclictest16593-21tune2fs21:40:150
20566992110,8cyclictest29693-21df23:50:150
2056999209,8cyclictest21980-21df_inode21:20:133
2056999209,8cyclictest21980-21df_inode21:20:133
2056999201,8cyclictest141rcu_preempt23:47:083
20569992014,4cyclictest53050irq/142-enp3s0-00:10:183
20569992014,4cyclictest19348-21grep22:05:273
20569992013,4cyclictest1131-21sh00:16:303
20569992012,6cyclictest31807-21cat22:15:163
20569992012,6cyclictest26650-21ntpq22:35:203
20569992012,4cyclictest34-21ksoftirqd/322:25:003
2056999201,16cyclictest14544-21ls21:15:003
20569992011,2cyclictest141rcu_preempt22:30:373
20568992014,4cyclictest4744-21sh22:19:462
20568992014,4cyclictest17552-21sh22:52:522
20568992014,4cyclictest17552-21sh22:52:522
20568992012,6cyclictest27199-21expr19:25:142
20568992012,6cyclictest18627-21sed22:05:212
20568992012,6cyclictest1726-21perl19:40:142
20568992012,6cyclictest1726-21perl19:40:142
20568992011,6cyclictest278-21systemd-journal19:25:002
20568992011,6cyclictest11455-21memory00:00:182
2056699202,12cyclictest18968-21ssh00:30:130
2056699201,6cyclictest4170-21apt-get21:55:120
20566992013,5cyclictest947-21seq19:39:230
20566992013,4cyclictest3292-21missed_timers23:30:190
20566992013,4cyclictest26990-21sh23:48:060
20566992012,6cyclictest18694-21apt-get20:20:110
20566992012,6cyclictest16906-21seq00:28:360
20566992012,6cyclictest1625-21diskstats19:40:130
20566992012,6cyclictest1625-21diskstats19:40:130
20566992012,6cyclictest13815-21seq20:08:400
20566992012,6cyclictest13514-21seq23:14:200
20566992012,6cyclictest10827-21awk23:35:280
20566992012,4cyclictest8718-21sed21:10:160
20566992011,7cyclictest6942-21python321:05:230
20566992011,6cyclictest26521-21seq19:21:320
20566992011,6cyclictest14582-21tune2fs20:10:140
20566992010,7cyclictest29954-21df20:45:140
220772190,1sleep30-21swapper/323:20:173
2056999198,2cyclictest34-21ksoftirqd/323:09:083
2056999193,9cyclictest24429-21apt-get22:10:143
2056999193,11cyclictest75-21kswapd023:50:133
2056999191,4cyclictest0-21swapper/321:33:533
20569991913,4cyclictest4385-21sh23:55:183
20569991913,4cyclictest29755-21cpuspeed_turbos21:50:143
20569991912,5cyclictest2728-21grep19:40:243
20569991912,5cyclictest2728-21grep19:40:243
20569991911,6cyclictest15378-21tune2fs21:15:163
2056899192,12cyclictest640-21snmpd20:55:122
20568991915,2cyclictest22959-21/usr/sbin/munin20:30:172
20568991915,2cyclictest22959-21/usr/sbin/munin20:30:172
20568991914,3cyclictest5728-21sh23:32:022
20568991914,3cyclictest10045-21sh22:47:062
20568991913,4cyclictest5929-21sh21:31:572
20568991913,4cyclictest29034-21sh23:00:522
20568991913,4cyclictest24692-21sh00:34:262
20568991912,5cyclictest9049-21seq22:22:192
20568991911,6cyclictest8169-21seq00:21:342
20568991911,6cyclictest4604-21grep23:55:202
20568991911,6cyclictest29656-21awk21:50:122
20568991911,6cyclictest25915-21basename20:35:162
20568991911,6cyclictest21801-21cat21:20:122
20568991911,6cyclictest21801-21cat21:20:122
20568991911,5cyclictest7672-21diskmemload22:59:112
20568991911,5cyclictest4006-21idleruntime19:45:152
20568991910,6cyclictest10652-21cat20:00:182
2056699199,7cyclictest20841-21apt-get20:25:110
2056699192,12cyclictest75-21kswapd022:40:110
20566991912,5cyclictest560-21dbus-daemon21:40:000
20566991912,5cyclictest29197-21seq20:42:510
20566991912,4cyclictest636-21runrttasks22:32:200
20566991912,4cyclictest21137-21sh21:19:590
20566991910,7cyclictest4008-21df23:55:150
20566991910,6cyclictest730-21cron23:05:000
2056999188,7cyclictest19017-21df20:20:133
2056999182,5cyclictest13360-21apt-get22:50:123
2056999182,5cyclictest13360-21apt-get22:50:123
2056999182,3cyclictest141rcu_preempt23:26:513
2056999182,11cyclictest5790-21taskset21:05:133
20569991813,3cyclictest13218-21swap20:05:243
20569991811,5cyclictest302-21which20:50:223
2056999181,14cyclictest15477-21kworker/u8:223:00:253
20569991810,6cyclictest17871-21fw_packets00:05:193
20569991810,6cyclictest11886-21cut20:05:013
2056899189,7cyclictest7044-21sed21:05:242
2056899189,7cyclictest20201-21vmstat20:20:252
2056899188,6cyclictest52950irq/141-enp3s0-00:05:142
20568991813,3cyclictest3978-21sh21:00:142
20568991813,3cyclictest31347-21sh21:26:392
20568991813,3cyclictest30322-21sh23:25:572
20568991813,3cyclictest25863-21sh00:10:312
20568991813,3cyclictest18261-21sh22:29:512
20568991813,3cyclictest17798-21sh23:40:392
20568991813,3cyclictest15171-21sh21:39:282
20568991812,4cyclictest25526-21sh22:35:092
20568991811,5cyclictest30952-21ls20:45:252
20568991811,5cyclictest22300-21grep20:25:242
20568991811,4cyclictest7672-21diskmemload22:30:572
20568991810,6cyclictest8977-21memory21:10:182
20568991810,6cyclictest30014-21date19:30:172
20568991810,6cyclictest15999-21cat23:15:252
20568991810,6cyclictest13467-21cat00:25:252
2056699189,7cyclictest560-21dbus-daemon21:50:010
2056699188,6cyclictest52750irq/139-enp3s0-21:30:190
20566991813,3cyclictest16536-21cpu20:15:110
20566991813,3cyclictest11632-21sh22:24:550
20566991812,4cyclictest18069-21sh23:17:100
20566991812,3cyclictest20976-21sh00:07:220
20566991811,5cyclictest1686-21cat20:55:130
20566991811,5cyclictest11977-21grep20:05:010
20566991811,4cyclictest7672-21diskmemload21:25:530
2056699181,12cyclictest3428-21apt-get19:45:110
20566991810,6cyclictest560-21dbus-daemon22:50:000
2056999178,7cyclictest25376-21perl19:20:143
2056999178,6cyclictest27065-21cpu19:25:133
2056999172,10cyclictest22898-21apt-get21:45:133
2056999171,6cyclictest278-21systemd-journal20:15:003
20569991712,3cyclictest7019-21unixbench_singl19:50:253
20569991711,4cyclictest10687-21timerwakeupswit23:35:273
20569991710,5cyclictest32243-21sed19:35:163
20569991710,5cyclictest24418-21awk20:30:263
20569991710,5cyclictest24418-21awk20:30:263
2056899179,6cyclictest27623-21expr20:40:132
20568991712,3cyclictest16391-21apt-config20:15:012
20568991711,4cyclictest22684-21sh23:20:242
20568991710,5cyclictest31959-21cat20:50:132
2056699179,6cyclictest16697-21ls23:40:180
20566991712,3cyclictest4300-21kernelversion21:00:170
2056699171,13cyclictest27385-21chrt19:25:150
2056999169,4cyclictest34-21ksoftirqd/319:55:113
2056999164,8cyclictest1489-21apt-get20:55:113
20569991610,4cyclictest34-21ksoftirqd/320:25:233
20569991610,3cyclictest34-21ksoftirqd/319:35:003
2056899169,5cyclictest5786-21expr19:50:132
20566991610,3cyclictest25137-21apt-key20:35:010
2056999159,4cyclictest3410-21basename19:45:093
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional