You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-29 - 16:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Wed Jul 29, 2026 12:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
139099920,52rtkit-daemon0-21swapper/207:09:392
396091226539,21sleep00-21swapper/007:07:360
396090526545,14sleep30-21swapper/307:07:313
396100126242,14sleep10-21swapper/107:08:411
40008102540,16sleep00-21swapper/009:00:160
40373152490,3sleep00-21swapper/010:45:010
40331152480,3sleep10-21swapper/110:30:221
40364822470,4sleep33056-21PK-Backend10:40:193
39852772470,2sleep20-21swapper/208:15:192
40643512390,3sleep20-21swapper/212:00:142
40643512390,3sleep20-21swapper/212:00:132
139099330,5rtkit-daemon2317-21gsd-power12:36:560
3961243992712,13cyclictest4000194-21unin-run09:00:002
396123999261,22cyclictest3981304-21/usr/sbin/munin08:05:141
139099260,4rtkit-daemon1389-21rtkit-daemon09:44:390
40444352250,6sleep342-21ksoftirqd/311:05:133
3961239992516,7cyclictest4012287-21cron09:35:001
3961239992414,8cyclictest4014023-21cron09:40:001
3961239992412,9cyclictest4024945-21cron10:09:591
3961246992315,6cyclictest4003637-21cron09:10:003
396124399239,12cyclictest3962893-21cron07:15:002
3961243992320,2cyclictest0-21swapper/210:14:212
396123999238,13cyclictest4015795-21unin-run09:45:001
3961236992315,6cyclictest4018883-21sensors_temp09:50:210
39757182220,4sleep33056-21PK-Backend07:50:013
3961246992214,6cyclictest3961811-21df_inode07:10:143
3961243992220,1cyclictest0-21swapper/210:08:262
3961243992220,1cyclictest0-21swapper/209:56:362
3961243992220,1cyclictest0-21swapper/208:25:582
3961243992218,3cyclictest0-21swapper/209:38:512
396124399220,21cyclictest0-21swapper/212:09:392
396124399220,21cyclictest0-21swapper/211:30:452
396124399220,21cyclictest0-21swapper/210:28:532
396124399220,20cyclictest0-21swapper/207:51:452
3961239992220,1cyclictest0-21swapper/111:22:041
3961239992220,1cyclictest0-21swapper/111:09:471
3961239992220,1cyclictest0-21swapper/110:27:351
3961239992220,1cyclictest0-21swapper/109:58:021
3961239992219,2cyclictest3056-21PK-Backend12:08:011
396123999220,20cyclictest4029555-21python310:20:191
3961236992220,1cyclictest0-21swapper/008:38:400
3961236992220,1cyclictest0-21swapper/008:15:480
3961236992220,1cyclictest0-21swapper/007:48:440
3961236992216,5cyclictest3987290-21wc08:20:210
3961236992214,6cyclictest436-20systemd-journal10:14:590
396123699220,20cyclictest4051216-21sort11:25:020
40709802210,2sleep00-21swapper/012:20:010
3961246992119,1cyclictest3056-21PK-Backend10:59:183
396124699210,20cyclictest3056-21PK-Backend07:17:593
396124699210,19cyclictest3056-21PK-Backend08:31:073
3961243992119,1cyclictest0-21swapper/212:37:522
3961243992119,1cyclictest0-21swapper/211:57:422
3961243992119,1cyclictest0-21swapper/211:52:422
3961243992119,1cyclictest0-21swapper/211:45:122
3961243992119,1cyclictest0-21swapper/211:38:162
3961243992119,1cyclictest0-21swapper/211:11:322
3961243992119,1cyclictest0-21swapper/211:09:032
3961243992119,1cyclictest0-21swapper/210:18:332
3961243992119,1cyclictest0-21swapper/209:26:342
3961243992119,1cyclictest0-21swapper/209:12:022
3961243992119,1cyclictest0-21swapper/209:12:012
3961243992119,1cyclictest0-21swapper/209:06:402
3961243992119,1cyclictest0-21swapper/207:45:162
3961243992119,1cyclictest0-21swapper/207:30:082
3961243992119,1cyclictest0-21swapper/207:19:272
3961243992112,5cyclictest28650irq/123-ahci[0008:00:212
3961243992110,8cyclictest1-21systemd10:50:002
396124399210,20cyclictest0-21swapper/212:24:192
396124399210,20cyclictest0-21swapper/212:16:012
396124399210,20cyclictest0-21swapper/210:22:112
396124399210,20cyclictest0-21swapper/209:50:142
396124399210,20cyclictest0-21swapper/208:12:532
396124399210,20cyclictest0-21swapper/207:26:292
396124399210,19cyclictest3056-21PK-Backend08:40:192
3961239992119,1cyclictest0-21swapper/112:10:311
3961239992119,1cyclictest0-21swapper/111:49:561
3961239992119,1cyclictest0-21swapper/111:33:461
3961239992119,1cyclictest0-21swapper/111:02:301
3961239992119,1cyclictest0-21swapper/110:45:281
3961239992119,1cyclictest0-21swapper/109:51:061
3961239992119,1cyclictest0-21swapper/109:20:241
3961239992119,1cyclictest0-21swapper/109:16:451
3961239992119,1cyclictest0-21swapper/108:39:541
3961239992119,1cyclictest0-21swapper/107:51:481
3961239992112,6cyclictest436-20systemd-journal10:59:591
3961239992112,6cyclictest3993480-21/usr/sbin/munin08:40:241
396123999210,20cyclictest0-21swapper/107:32:431
396123999210,1cyclictest0-21swapper/110:37:561
3961236992119,1cyclictest3056-21PK-Backend11:55:380
3961236992119,1cyclictest0-21swapper/012:31:190
3961236992119,1cyclictest0-21swapper/011:11:570
3961236992119,1cyclictest0-21swapper/009:06:050
3961236992119,1cyclictest0-21swapper/007:58:170
3961236992115,5cyclictest3998460-21systemd-run08:55:000
396123699210,20cyclictest3990152-21diskstats08:30:150
396123699210,20cyclictest0-21swapper/011:33:330
396123699210,20cyclictest0-21swapper/010:34:250
396123699210,20cyclictest0-21swapper/010:29:190
139099210,5rtkit-daemon1788643-21eadedCompositor11:50:201
139099210,4rtkit-daemon1389-21rtkit-daemon11:11:201
139099210,15rtkit-daemon0-21swapper/207:43:282
3961246992018,1cyclictest3056-21PK-Backend09:42:593
3961246992018,1cyclictest0-21swapper/308:16:543
3961246992017,2cyclictest3056-21PK-Backend07:33:073
396124699200,1cyclictest3056-21PK-Backend09:36:173
396124699200,19cyclictest3056-21PK-Backend12:29:563
396124699200,19cyclictest3056-21PK-Backend09:51:033
396124699200,19cyclictest3056-21PK-Backend08:13:503
396124699200,19cyclictest3056-21PK-Backend07:28:203
396124699200,19cyclictest0-21swapper/312:05:283
396124699200,19cyclictest0-21swapper/309:26:293
396124699200,18cyclictest3056-21PK-Backend10:20:043
3961243992018,1cyclictest0-21swapper/212:32:042
3961243992018,1cyclictest0-21swapper/211:02:552
3961243992018,1cyclictest0-21swapper/210:56:452
3961243992018,1cyclictest0-21swapper/209:48:192
3961243992018,1cyclictest0-21swapper/207:55:302
3961243992018,1cyclictest0-21swapper/207:38:122
3961243992015,4cyclictest3992973-21latency_hist08:40:002
396124399200,1cyclictest3056-21PK-Backend08:05:232
396124399200,1cyclictest0-21swapper/211:20:112
396124399200,1cyclictest0-21swapper/209:00:122
396124399200,19cyclictest0-21swapper/212:11:352
396124399200,19cyclictest0-21swapper/209:41:222
396123999209,9cyclictest1936-21dbus-daemon12:28:501
396123999209,8cyclictest883-21dbus-daemon12:25:001
3961239992018,1cyclictest3056-21PK-Backend10:52:091
3961239992018,1cyclictest0-21swapper/112:31:531
3961239992018,1cyclictest0-21swapper/111:39:211
3961239992018,1cyclictest0-21swapper/109:11:241
3961239992018,1cyclictest0-21swapper/109:11:241
3961239992018,1cyclictest0-21swapper/108:34:401
3961239992018,1cyclictest0-21swapper/107:47:291
3961239992018,1cyclictest0-21swapper/107:17:211
3961239992015,4cyclictest4048655-21switchtime11:15:221
3961239992013,6cyclictest3988986-21seq08:25:231
3961239992012,6cyclictest3972126-21munin-plugin-st07:40:001
396123999201,14cyclictest3998697-21apt08:55:111
3961239992010,7cyclictest4062534-21cpuspeed_turbos11:55:131
396123999200,1cyclictest0-21swapper/109:25:111
396123999200,1cyclictest0-21swapper/108:00:411
396123999200,19cyclictest0-21swapper/111:28:121
396123999200,19cyclictest0-21swapper/109:46:401
396123999200,18cyclictest0-21swapper/108:15:131
3961236992018,1cyclictest0-21swapper/012:00:440
3961236992018,1cyclictest0-21swapper/012:00:440
3961236992018,1cyclictest0-21swapper/011:15:210
3961236992013,5cyclictest4005351-21cron09:15:000
3961236992013,5cyclictest4005351-21cron09:14:590
3961236992012,6cyclictest436-20systemd-journal07:45:010
3961236992012,6cyclictest4050963-21apt-get11:25:010
3961236992012,6cyclictest3982838-21apt08:10:110
396123699200,5cyclictest4060158-21grep11:50:000
396123699200,19cyclictest0-21swapper/009:22:400
396123699200,19cyclictest0-21swapper/008:55:430
139099200,17rtkit-daemon894-21irqbalance12:02:201
139099200,17rtkit-daemon894-21irqbalance12:02:201
139099200,14rtkit-daemon0-21swapper/208:52:102
39767042190,4sleep30-21swapper/307:50:203
39623112190,4chrt0-21swapper/107:10:201
3961246991917,1cyclictest3056-21PK-Backend11:34:333
3961246991917,1cyclictest3056-21PK-Backend11:02:563
3961246991917,1cyclictest3056-21PK-Backend10:33:483
3961246991917,1cyclictest3056-21PK-Backend09:15:143
3961246991917,1cyclictest3056-21PK-Backend08:52:153
3961246991917,1cyclictest0-21swapper/312:02:593
3961246991917,1cyclictest0-21swapper/312:02:583
396124699191,6cyclictest4011575-21ntp_states09:30:203
3961246991916,2cyclictest3056-21PK-Backend12:31:513
3961246991916,2cyclictest3056-21PK-Backend11:46:153
3961246991916,2cyclictest3056-21PK-Backend10:37:273
396124699191,16cyclictest82332-21update-manager09:55:223
396124699190,1cyclictest3056-21PK-Backend11:37:363
396124699190,1cyclictest3056-21PK-Backend11:21:413
396124699190,1cyclictest3056-21PK-Backend11:10:463
396124699190,1cyclictest3056-21PK-Backend10:15:163
396124699190,1cyclictest3056-21PK-Backend08:37:293
396124699190,1cyclictest3056-21PK-Backend08:29:103
396124699190,1cyclictest3056-21PK-Backend07:38:423
396124699190,1cyclictest0-21swapper/310:50:533
396124699190,18cyclictest3056-21PK-Backend12:17:583
396124699190,18cyclictest3056-21PK-Backend11:43:453
396124699190,18cyclictest3056-21PK-Backend08:46:003
396124699190,18cyclictest0-21swapper/312:10:283
396124699190,18cyclictest0-21swapper/311:55:283
396124699190,18cyclictest0-21swapper/310:47:483
396124699190,17cyclictest3056-21PK-Backend11:52:233
396124699190,17cyclictest3056-21PK-Backend11:18:033
3961243991917,1cyclictest0-21swapper/211:25:112
3961243991917,1cyclictest0-21swapper/211:17:062
3961243991917,1cyclictest0-21swapper/209:33:172
3961243991917,1cyclictest0-21swapper/209:15:402
3961243991916,2cyclictest0-21swapper/210:30:502
3961243991911,6cyclictest4039538-21cat10:50:162
396124399190,1cyclictest0-21swapper/210:00:152
396124399190,1cyclictest0-21swapper/210:00:142
396123999198,9cyclictest4058150-21apt11:45:001
3961239991917,1cyclictest0-21swapper/112:36:191
3961239991917,1cyclictest0-21swapper/112:15:301
3961239991917,1cyclictest0-21swapper/107:29:041
3961239991916,2cyclictest4003645-21idleruntime-cro09:09:591
3961239991913,5cyclictest3965832-21if_enp2s007:20:171
396123999190,1cyclictest0-21swapper/108:24:271
396123999190,18cyclictest3056-21PK-Backend07:56:561
396123999190,18cyclictest0-21swapper/110:43:321
396123999190,18cyclictest0-21swapper/108:45:561
3961236991916,2cyclictest0-21swapper/012:05:170
3961236991915,3cyclictest3968605-21idleruntime-cro07:29:590
3961236991914,3cyclictest4005725-21cpuspeed_turbos09:15:110
3961236991914,3cyclictest3988623-21missed_timers08:25:190
3961236991913,5cyclictest3970362-21systemd-run07:35:010
3961236991912,6cyclictest4009252-21expr09:25:120
3961236991911,6cyclictest4013279-21ntpq09:35:200
396123699190,18cyclictest0-21swapper/007:17:350
139099190,13rtkit-daemon0-21swapper/208:21:042
3961246991816,1cyclictest3056-21PK-Backend12:22:253
3961246991816,1cyclictest3056-21PK-Backend09:48:333
3961246991815,2cyclictest3056-21PK-Backend12:36:173
3961246991812,5cyclictest4000172-21cron09:00:003
396124699180,16cyclictest3979844-21/usr/sbin/munin08:00:163
3961243991816,1cyclictest0-21swapper/209:22:562
396124399180,6cyclictest0-21swapper/210:35:192
396123999189,5cyclictest4000915-21kernelversion09:00:171
3961239991812,5cyclictest4027143-21grep10:15:131
3961239991811,6cyclictest3972691-21cstates07:40:131
3961236991814,2cyclictest4056674-21acpi11:40:110
3961236991811,5cyclictest4027978-21cat10:15:220
3961236991810,6cyclictest3996528-21pool-udisksd08:48:030
40308492170,3sleep33056-21PK-Backend10:25:163
3961246991715,1cyclictest3056-21PK-Backend10:01:253
3961246991715,1cyclictest3056-21PK-Backend10:01:253
3961246991715,1cyclictest3056-21PK-Backend09:21:093
3961246991715,1cyclictest3056-21PK-Backend08:23:023
3961246991711,5cyclictest3978736-21taskset07:55:253
396124699170,15cyclictest4023546-21grep10:05:123
3961243991715,1cyclictest0-21swapper/211:40:122
3961243991715,1cyclictest0-21swapper/208:31:462
396123999179,6cyclictest4022359-21sensors_temp10:00:221
396123999179,6cyclictest4022359-21sensors_temp10:00:221
396123999179,6cyclictest3997642-21uname08:50:191
396123999179,5cyclictest3983288-21/usr/sbin/munin08:10:151
3961239991715,1cyclictest0-21swapper/110:14:591
396123699179,6cyclictest3971697-21seq07:35:310
396123699179,6cyclictest1-21systemd12:30:000
3961236991715,1cyclictest3056-21PK-Backend10:51:300
3961236991715,1cyclictest0-21swapper/008:05:210
3961236991715,1cyclictest0-21swapper/007:22:010
3961236991713,3cyclictest4033901-21cpu10:35:120
3961236991713,3cyclictest3976985-21uptime07:50:230
3961236991713,3cyclictest0-21swapper/011:01:090
3961236991712,3cyclictest4022274-21open_inodes10:00:220
3961236991712,3cyclictest4022274-21open_inodes10:00:210
3961236991711,5cyclictest4011245-21basename09:30:170
3961236991710,5cyclictest4067587-21find12:10:120
3961236991710,5cyclictest3994703-21seq08:41:500
3961246991613,2cyclictest3056-21PK-Backend08:07:353
3961243991615,1cyclictest0-21swapper/212:25:412
396123699169,5cyclictest4030333-21cut10:25:010
3961236991611,3cyclictest4055352-21sh11:35:160
3961236991610,5cyclictest4016564-21sed09:45:170
396123699160,15cyclictest0-21swapper/010:07:010
39727642150,4sleep33056-21PK-Backend07:40:153
3961246991513,1cyclictest3056-21PK-Backend11:27:503
396124699150,13cyclictest3993480-21/usr/sbin/munin08:40:183
396124399159,4cyclictest3996743-21sort08:50:012
396124399158,5cyclictest4036483-21sensors_temp10:40:192
396123699159,5cyclictest4060658-21cat11:50:140
396123699158,6cyclictest883-21dbus-daemon08:05:000
396123699158,6cyclictest4071824-21open_files12:20:200
396123699157,6cyclictest4044650-21df11:05:130
396123699154,3cyclictest141rcu_preempt10:50:000
396123699152,10cyclictest4040826-21apt10:55:120
3961236991512,2cyclictest4021233-21latency_hist10:00:020
396124399149,4cyclictest3966990-21sed07:25:012
3961236991413,1cyclictest0-21swapper/007:10:520
396124699139,3cyclictest978-21snmpd07:22:353
396124699139,3cyclictest3056-21PK-Backend09:10:293
396124699139,3cyclictest3056-21PK-Backend09:10:293
396124699128,3cyclictest3056-21PK-Backend09:00:543
396124699120,11cyclictest3056-21PK-Backend10:11:393
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional