You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-25 - 01:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Mon Aug 24, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
181348221100,4sleep13056-21PK-Backend09:45:001
175800027738,14sleep10-21swapper/107:08:151
175805126443,14sleep20-21swapper/207:08:552
175767126341,14sleep30-21swapper/307:05:273
17756872620,9sleep013-21ksoftirqd/007:55:160
175798726242,14sleep00-21swapper/007:08:050
18381132480,2sleep33056-21PK-Backend10:55:003
17930652400,5sleep10-21swapper/108:45:181
18385722370,3sleep33056-21PK-Backend10:55:123
1758376992712,13cyclictest1864085-21cron12:10:002
1758366992622,3cyclictest161rcuc/010:03:080
139099260,18rtkit-daemon0-21swapper/008:20:150
18702872250,2sleep20-21swapper/212:25:172
1758372992512,11cyclictest1842311-21irqstats11:05:171
175836699258,9cyclictest13-21ksoftirqd/007:40:000
1758366992516,6cyclictest1783527-21cron08:20:010
1758366992514,8cyclictest883-21dbus-daemon12:40:000
175837699240,23cyclictest1830806-21chrt10:30:332
1758372992417,6cyclictest1874886-21systemd12:40:001
1758366992421,2cyclictest161rcuc/011:54:430
1758366992421,2cyclictest161rcuc/008:56:380
175837699231,20cyclictest1832890-21cron10:40:002
175836699236,6cyclictest13-21ksoftirqd/010:10:130
175836699233,7cyclictest13-21ksoftirqd/011:25:210
175836699233,10cyclictest13-21ksoftirqd/007:55:000
1758366992320,2cyclictest161rcuc/011:59:250
1758366992312,8cyclictest883-21dbus-daemon09:55:000
139099230,16rtkit-daemon3056-21PK-Backend10:13:271
139099230,15rtkit-daemon0-21swapper/008:26:530
175838099227,3cyclictest0-21swapper/309:35:113
175838099227,3cyclictest0-21swapper/309:35:103
1758380992214,6cyclictest1821966-21apt-get10:10:013
1758376992215,6cyclictest1813337-21systemctl09:45:002
175837699220,21cyclictest0-21swapper/212:13:492
1758372992220,1cyclictest3056-21PK-Backend10:45:011
1758372992219,2cyclictest3056-21PK-Backend09:10:321
175837299220,21cyclictest3056-21PK-Backend11:39:231
175837299220,21cyclictest3056-21PK-Backend11:39:231
175837299220,20cyclictest3056-21PK-Backend09:09:311
1758366992220,1cyclictest0-21swapper/012:13:120
1758366992220,1cyclictest0-21swapper/011:41:080
1758366992220,1cyclictest0-21swapper/010:47:480
1758366992220,1cyclictest0-21swapper/010:06:460
1758366992220,1cyclictest0-21swapper/009:42:380
1758366992220,1cyclictest0-21swapper/009:06:150
1758366992219,2cyclictest161rcuc/011:39:380
1758366992219,2cyclictest161rcuc/011:39:380
1758366992213,7cyclictest1827441-21munin-run10:25:000
1758366992212,7cyclictest436-20systemd-journal12:20:000
139099220,15rtkit-daemon3056-21PK-Backend07:33:261
18560712210,4sleep33056-21PK-Backend11:45:123
18081302210,2sleep00-21swapper/009:30:120
1758380992114,6cyclictest1826748-21pluginstate10:20:193
1758380992112,7cyclictest1808497-21diskstats09:30:143
175838099210,8cyclictest1806965-21memory09:25:173
175838099210,20cyclictest3056-21PK-Backend08:40:543
175838099210,19cyclictest3056-21PK-Backend12:10:363
175838099210,19cyclictest3056-21PK-Backend08:30:063
175838099210,19cyclictest3056-21PK-Backend08:15:203
1758376992117,2cyclictest18114432sleep209:38:282
1758376992117,2cyclictest18114432sleep209:38:282
1758376992114,5cyclictest1822472-21sshd10:10:092
1758376992113,6cyclictest1834036-21python310:40:222
1758376992112,7cyclictest436-20systemd-journal11:34:592
1758372992119,1cyclictest3056-21PK-Backend08:02:121
1758372992119,1cyclictest0-21swapper/112:33:031
1758372992119,1cyclictest0-21swapper/112:24:521
1758372992119,1cyclictest0-21swapper/111:58:491
1758372992119,1cyclictest0-21swapper/111:53:501
1758372992119,1cyclictest0-21swapper/111:43:351
1758372992119,1cyclictest0-21swapper/111:24:371
1758372992119,1cyclictest0-21swapper/111:19:561
1758372992119,1cyclictest0-21swapper/110:52:591
1758372992118,2cyclictest3056-21PK-Backend09:33:381
1758372992118,2cyclictest3056-21PK-Backend07:53:121
1758372992118,2cyclictest3056-21PK-Backend07:22:161
175836699218,11cyclictest1781835-21unin-run08:15:010
1758366992119,1cyclictest3056-21PK-Backend09:45:430
1758366992119,1cyclictest0-21swapper/012:27:530
1758366992119,1cyclictest0-21swapper/012:22:380
1758366992119,1cyclictest0-21swapper/011:17:490
1758366992119,1cyclictest0-21swapper/011:14:450
1758366992119,1cyclictest0-21swapper/010:19:290
1758366992119,1cyclictest0-21swapper/009:27:580
1758366992119,1cyclictest0-21swapper/009:11:570
1758366992119,1cyclictest0-21swapper/009:02:230
1758366992119,1cyclictest0-21swapper/008:39:320
1758366992119,1cyclictest0-21swapper/008:09:310
1758366992119,1cyclictest0-21swapper/007:29:010
1758366992119,1cyclictest0-21swapper/007:18:480
1758366992118,2cyclictest161rcuc/011:49:480
175836699210,20cyclictest0-21swapper/010:36:340
175836699210,20cyclictest0-21swapper/009:57:460
139099210,15rtkit-daemon3056-21PK-Backend08:10:451
1758380992018,1cyclictest0-21swapper/308:07:553
1758380992017,2cyclictest3056-21PK-Backend08:57:513
1758380992012,6cyclictest883-21dbus-daemon10:30:003
1758380992012,6cyclictest1775890-21ntp_states07:55:193
1758380992012,6cyclictest1758743-21apt07:10:123
175838099200,1cyclictest3056-21PK-Backend11:43:113
175838099200,1cyclictest3056-21PK-Backend11:07:443
175838099200,1cyclictest0-21swapper/312:20:363
175838099200,1cyclictest0-21swapper/308:03:433
175838099200,19cyclictest3056-21PK-Backend12:28:133
175838099200,19cyclictest3056-21PK-Backend08:54:123
175838099200,19cyclictest0-21swapper/311:38:123
175838099200,19cyclictest0-21swapper/311:38:113
175838099200,19cyclictest0-21swapper/311:32:303
175838099200,19cyclictest0-21swapper/309:52:463
175838099200,19cyclictest0-21swapper/309:43:343
175838099200,18cyclictest3056-21PK-Backend10:00:443
1758376992018,1cyclictest0-21swapper/212:35:252
1758376992018,1cyclictest0-21swapper/211:37:312
1758376992018,1cyclictest0-21swapper/211:37:312
1758376992018,1cyclictest0-21swapper/211:20:282
1758376992018,1cyclictest0-21swapper/211:18:262
1758376992018,1cyclictest0-21swapper/207:47:352
1758376992017,2cyclictest0-21swapper/210:45:212
1758376992013,5cyclictest1759185-21cat07:10:162
175837699200,1cyclictest0-21swapper/210:25:122
175837699200,1cyclictest0-21swapper/209:26:522
175837699200,1cyclictest0-21swapper/208:46:302
175837699200,19cyclictest0-21swapper/211:48:332
175837699200,19cyclictest0-21swapper/211:00:272
175837699200,19cyclictest0-21swapper/210:57:232
175837699200,19cyclictest0-21swapper/210:20:532
175837699200,19cyclictest0-21swapper/210:17:362
175837699200,19cyclictest0-21swapper/209:18:202
175837699200,19cyclictest0-21swapper/208:42:512
175837699200,19cyclictest0-21swapper/208:36:022
175837699200,19cyclictest0-21swapper/208:27:382
175837699200,19cyclictest0-21swapper/208:20:002
175837699200,19cyclictest0-21swapper/208:09:122
175837699200,19cyclictest0-21swapper/207:55:002
175837699200,19cyclictest0-21swapper/207:34:392
1758372992018,1cyclictest3056-21PK-Backend10:17:581
1758372992018,1cyclictest3056-21PK-Backend10:05:271
1758372992018,1cyclictest3056-21PK-Backend07:28:451
1758372992018,1cyclictest0-21swapper/112:18:031
1758372992018,1cyclictest0-21swapper/111:11:581
1758372992018,1cyclictest0-21swapper/110:03:241
1758372992018,1cyclictest0-21swapper/109:56:291
1758372992018,1cyclictest0-21swapper/109:00:121
1758372992017,2cyclictest3056-21PK-Backend09:23:291
175837299200,19cyclictest3056-21PK-Backend09:18:501
175837299200,19cyclictest3056-21PK-Backend08:25:091
175837299200,19cyclictest0-21swapper/112:27:291
175837299200,19cyclictest0-21swapper/111:47:001
175837299200,19cyclictest0-21swapper/111:02:591
175837299200,19cyclictest0-21swapper/110:55:031
175836699204,7cyclictest13-21ksoftirqd/008:34:590
175836699202,7cyclictest1759081-21expr07:10:140
1758366992018,1cyclictest0-21swapper/012:32:390
1758366992018,1cyclictest0-21swapper/012:09:470
1758366992018,1cyclictest0-21swapper/011:30:140
1758366992018,1cyclictest0-21swapper/010:27:060
1758366992018,1cyclictest0-21swapper/008:43:590
1758366992018,1cyclictest0-21swapper/008:02:550
1758366992017,2cyclictest161rcuc/010:52:390
1758366992017,2cyclictest161rcuc/008:45:290
175836699201,6cyclictest1860872-21/usr/sbin/munin12:00:220
1758366992016,3cyclictest1848483-21cron11:25:000
175836699200,19cyclictest0-21swapper/011:03:430
175836699200,19cyclictest0-21swapper/010:43:420
175836699200,19cyclictest0-21swapper/009:37:320
175836699200,19cyclictest0-21swapper/009:37:310
175836699200,19cyclictest0-21swapper/007:23:070
139099200,15rtkit-daemon3056-21PK-Backend10:21:211
139099200,14rtkit-daemon3056-21PK-Backend10:33:381
139099200,13rtkit-daemon3056-21PK-Backend07:41:231
139099200,13rtkit-daemon0-21swapper/108:18:341
18546112190,4sleep23056-21PK-Backend11:40:192
1758380991917,1cyclictest0-21swapper/310:47:553
1758380991917,1cyclictest0-21swapper/310:10:453
1758380991917,1cyclictest0-21swapper/307:52:553
1758380991917,1cyclictest0-21swapper/307:35:103
1758380991917,1cyclictest0-21swapper/307:22:203
1758380991914,3cyclictest1761697-21ntp_states07:15:173
175838099191,16cyclictest1800314-21python309:05:203
175838099190,1cyclictest3056-21PK-Backend12:16:243
175838099190,1cyclictest3056-21PK-Backend11:26:013
175838099190,1cyclictest3056-21PK-Backend11:18:313
175838099190,1cyclictest3056-21PK-Backend09:58:413
175838099190,1cyclictest3056-21PK-Backend08:35:543
175838099190,1cyclictest0-21swapper/309:48:413
175838099190,1cyclictest0-21swapper/307:31:123
175838099190,18cyclictest0-21swapper/311:55:013
175837699197,11cyclictest34-21ksoftirqd/209:05:002
1758376991917,1cyclictest0-21swapper/212:16:252
1758376991917,1cyclictest0-21swapper/211:27:582
1758376991917,1cyclictest0-21swapper/208:56:382
1758376991917,1cyclictest0-21swapper/208:51:102
1758376991915,3cyclictest978-21snmpd07:41:572
1758376991912,6cyclictest1764837-21ls07:25:172
175837699190,1cyclictest0-21swapper/212:00:232
175837699190,1cyclictest0-21swapper/210:02:482
175837699190,1cyclictest0-21swapper/209:59:442
175837699190,1cyclictest0-21swapper/209:50:452
175837699190,1cyclictest0-21swapper/209:34:222
175837699190,1cyclictest0-21swapper/209:08:472
175837699190,1cyclictest0-21swapper/208:32:372
175837699190,18cyclictest0-21swapper/212:30:172
175837699190,18cyclictest0-21swapper/211:07:232
1758372991917,1cyclictest3056-21PK-Backend11:33:081
1758372991917,1cyclictest3056-21PK-Backend10:40:561
1758372991917,1cyclictest3056-21PK-Backend07:15:131
1758372991917,1cyclictest0-21swapper/112:13:501
1758372991917,1cyclictest0-21swapper/111:25:371
1758372991912,5cyclictest1813785-21cat09:45:131
1758372991912,5cyclictest1796583-21cat08:55:171
175837299190,1cyclictest0-21swapper/108:42:471
175837299190,1cyclictest0-21swapper/108:35:111
1758366991915,3cyclictest161rcuc/008:51:050
175836699190,18cyclictest0-21swapper/007:34:220
139099190,13rtkit-daemon0-21swapper/112:08:121
1758380991816,1cyclictest3056-21PK-Backend12:32:123
1758380991816,1cyclictest3056-21PK-Backend08:49:473
1758380991816,1cyclictest0-21swapper/308:24:183
1758380991814,3cyclictest978-21snmpd09:20:013
1758380991811,6cyclictest1802236-21sed09:10:223
175838099181,15cyclictest1831197-21munin-run10:35:003
175838099180,6cyclictest1833170-21apt10:40:133
175838099180,1cyclictest0-21swapper/309:21:233
175838099180,16cyclictest1832180-21ntp_states10:35:183
1758376991816,1cyclictest0-21swapper/212:22:272
1758376991816,1cyclictest0-21swapper/209:24:022
1758376991816,1cyclictest0-21swapper/208:22:382
1758376991815,2cyclictest0-21swapper/207:58:252
1758376991813,4cyclictest34-21ksoftirqd/207:35:212
1758372991816,1cyclictest3056-21PK-Backend09:38:301
1758372991816,1cyclictest3056-21PK-Backend09:38:301
1758372991816,1cyclictest3056-21PK-Backend09:29:591
1758372991816,1cyclictest3056-21PK-Backend07:35:421
1758372991815,2cyclictest3056-21PK-Backend08:08:001
1758372991811,5cyclictest1784907-21seq08:20:261
175836699188,3cyclictest13-21ksoftirqd/007:47:200
1758366991816,1cyclictest0-21swapper/009:17:370
1758366991815,2cyclictest161rcuc/011:07:220
1758366991813,3cyclictest1805321-21needreboot09:20:160
18578052170,4sleep33056-21PK-Backend11:50:143
1758380991715,1cyclictest3056-21PK-Backend12:38:413
1758380991714,2cyclictest3056-21PK-Backend12:04:013
1758380991713,3cyclictest42-21ksoftirqd/308:10:123
1758380991711,5cyclictest1772563-21wc07:45:223
1758380991711,4cyclictest1825102-21python310:15:183
1758380991710,5cyclictest918-21rs:main3
175837699177,8cyclictest883-21dbus-daemon10:10:002
1758376991715,1cyclictest0-21swapper/211:50:222
1758376991712,4cyclictest1777058-21grep08:00:132
1758376991712,3cyclictest978-21snmpd07:15:052
1758376991712,3cyclictest1860635-21idleruntime-cro12:00:002
1758376991711,4cyclictest2156-21gnome-shell09:10:092
1758376991711,4cyclictest1763068-21sh07:20:142
1758376991710,5cyclictest1814225-21grep09:45:192
1758372991715,1cyclictest3056-21PK-Backend10:35:491
1758372991715,1cyclictest3056-21PK-Backend09:50:341
1758372991712,4cyclictest1794801-21cat08:50:191
1758372991711,4cyclictest1861085-21expr12:00:131
1758372991710,6cyclictest1787863-21memory08:30:181
1758366991714,2cyclictest161rcuc/010:59:330
1758366991714,2cyclictest161rcuc/010:32:320
1758380991614,1cyclictest0-21swapper/309:02:303
1758380991612,3cyclictest1863212-21missed_timers12:05:183
1758380991612,3cyclictest1770806-21python307:40:183
175838099161,14cyclictest1785546-21apt08:25:123
1758376991610,5cyclictest1845028-21cron11:15:002
175837699160,14cyclictest1838095-21perf10:55:002
175836699160,5cyclictest13-21ksoftirqd/007:40:110
1758380991513,1cyclictest3056-21PK-Backend11:20:343
175838099150,5cyclictest3056-21PK-Backend11:10:203
175837699151,11cyclictest1780346-21apt08:10:122
1758372991510,4cyclictest1829511-21cut10:30:011
175837299144,2cyclictest141rcu_preempt08:00:001
175837299144,2cyclictest141rcu_preempt07:10:131
1758380991311,1cyclictest3056-21PK-Backend11:03:513
1758372991310,2cyclictest3056-21PK-Backend07:45:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional