You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-22 - 17:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Mon Jun 22, 2026 12:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
399907121240,2sleep23056-21PK-Backend10:25:122
400674621000,3sleep13056-21PK-Backend10:45:161
40460332990,2sleep13056-21PK-Backend12:35:201
392879027650,18sleep30-21swapper/307:05:243
392812127147,15sleep00-21swapper/007:05:140
392915726645,15sleep20-21swapper/207:07:042
392922826243,13sleep10-21swapper/107:07:591
40310632600,4sleep13056-21PK-Backend11:55:141
40384672400,3sleep30-21swapper/312:15:173
40262302290,4sleep00-21swapper/011:40:170
39358412280,8sleep234-21ksoftirqd/207:25:162
39304732270,3sleep126-21ksoftirqd/107:10:191
39960522230,11sleep013-21ksoftirqd/010:15:160
39960522230,11sleep013-21ksoftirqd/010:15:160
392951199237,14cyclictest4018978-21netstat11:20:153
392950899237,2cyclictest141rcu_preempt11:45:102
3929501992312,9cyclictest3930472-21ntp_states07:10:190
392951199220,20cyclictest3056-21PK-Backend08:34:213
3929508992220,1cyclictest0-21swapper/209:19:362
3929508992220,1cyclictest0-21swapper/208:45:092
3929508992220,1cyclictest0-21swapper/207:13:292
392950899220,20cyclictest4043365-21apt12:30:112
3929506992214,5cyclictest1095-21runrttasks07:15:111
3929501992220,1cyclictest0-21swapper/012:36:450
3929501992220,1cyclictest0-21swapper/011:05:000
3929501992220,1cyclictest0-21swapper/009:55:510
3929501992220,1cyclictest0-21swapper/009:12:520
3929501992220,1cyclictest0-21swapper/009:03:530
3929501992220,1cyclictest0-21swapper/007:27:270
139099220,16rtkit-daemon3056-21PK-Backend11:23:571
139099220,14rtkit-daemon0-21swapper/008:40:020
39803272210,4chrt3056-21PK-Backend09:30:170
3929511992118,2cyclictest0-21swapper/308:48:213
3929511992113,6cyclictest4030696-21apt11:55:123
3929511992113,6cyclictest3979309-21cron09:30:003
392951199210,20cyclictest0-21swapper/312:09:103
392951199210,20cyclictest0-21swapper/311:29:293
392951199210,19cyclictest3056-21PK-Backend10:20:203
392951199210,19cyclictest3056-21PK-Backend09:51:203
3929508992119,1cyclictest3056-21PK-Backend10:35:272
3929508992119,1cyclictest0-21swapper/212:36:132
3929508992119,1cyclictest0-21swapper/212:01:182
3929508992119,1cyclictest0-21swapper/211:58:002
3929508992119,1cyclictest0-21swapper/211:39:552
3929508992119,1cyclictest0-21swapper/209:36:392
3929508992119,1cyclictest0-21swapper/209:22:132
3929508992119,1cyclictest0-21swapper/208:52:132
3929508992119,1cyclictest0-21swapper/208:36:572
3929508992119,1cyclictest0-21swapper/208:00:542
392950899210,20cyclictest0-21swapper/211:51:252
392950899210,20cyclictest0-21swapper/211:23:192
392950899210,20cyclictest0-21swapper/210:13:102
392950899210,20cyclictest0-21swapper/210:13:102
392950899210,20cyclictest0-21swapper/209:50:042
392950899210,20cyclictest0-21swapper/209:46:062
3929506992113,6cyclictest3951327-21apt-get08:09:591
392950199212,1cyclictest0-21swapper/008:50:080
3929501992119,1cyclictest0-21swapper/012:26:520
3929501992119,1cyclictest0-21swapper/012:15:160
3929501992119,1cyclictest0-21swapper/011:58:270
3929501992119,1cyclictest0-21swapper/011:49:210
3929501992119,1cyclictest0-21swapper/011:34:410
3929501992119,1cyclictest0-21swapper/011:05:340
3929501992119,1cyclictest0-21swapper/010:37:080
3929501992119,1cyclictest0-21swapper/010:27:420
3929501992119,1cyclictest0-21swapper/010:22:140
3929501992119,1cyclictest0-21swapper/010:05:310
3929501992119,1cyclictest0-21swapper/009:36:250
3929501992119,1cyclictest0-21swapper/009:28:200
3929501992119,1cyclictest0-21swapper/009:19:010
3929501992119,1cyclictest0-21swapper/008:36:490
3929501992119,1cyclictest0-21swapper/008:19:390
3929501992119,1cyclictest0-21swapper/007:23:290
3929501992119,1cyclictest0-21swapper/007:17:200
392950199210,20cyclictest0-21swapper/007:33:290
139099210,14rtkit-daemon3056-21PK-Backend12:34:581
139099210,14rtkit-daemon3056-21PK-Backend08:49:411
3929511992018,1cyclictest3056-21PK-Backend08:26:173
3929511992017,2cyclictest0-21swapper/307:33:243
392951199201,6cyclictest40042902grep10:40:113
3929511992012,6cyclictest4022631-21python311:30:223
3929511992012,6cyclictest4011755-21fw_packets11:00:143
392951199200,1cyclictest3056-21PK-Backend09:24:173
392951199200,1cyclictest0-21swapper/312:31:343
392951199200,19cyclictest3056-21PK-Backend12:24:583
392951199200,19cyclictest3056-21PK-Backend10:16:353
392951199200,19cyclictest3056-21PK-Backend10:16:353
392951199200,19cyclictest3056-21PK-Backend07:41:023
392951199200,19cyclictest0-21swapper/310:49:343
392951199200,19cyclictest0-21swapper/308:16:043
392951199200,19cyclictest0-21swapper/308:09:003
392951199200,19cyclictest0-21swapper/308:01:233
392951199200,19cyclictest0-21swapper/307:10:133
392951199200,18cyclictest3056-21PK-Backend07:20:263
392950899203,15cyclictest3944146-21grep07:50:002
3929508992018,1cyclictest3056-21PK-Backend07:15:052
3929508992018,1cyclictest0-21swapper/212:25:312
3929508992018,1cyclictest0-21swapper/211:34:012
3929508992018,1cyclictest0-21swapper/211:28:402
3929508992018,1cyclictest0-21swapper/211:17:592
3929508992018,1cyclictest0-21swapper/210:45:072
3929508992018,1cyclictest0-21swapper/208:34:342
3929508992018,1cyclictest0-21swapper/208:20:562
3929508992018,1cyclictest0-21swapper/208:20:552
3929508992018,1cyclictest0-21swapper/208:11:292
3929508992018,1cyclictest0-21swapper/207:57:582
3929508992017,2cyclictest917-21in:imuxsock07:32:292
3929508992015,3cyclictest978-21snmpd08:40:062
3929508992013,5cyclictest3996908-21cron10:20:012
3929508992013,5cyclictest3996908-21cron10:20:002
3929508992012,6cyclictest4035163-21ntpq12:05:182
3929508992012,6cyclictest3951195-21sshd08:08:292
392950899200,1cyclictest0-21swapper/212:13:422
392950899200,19cyclictest0-21swapper/212:18:562
392950899200,19cyclictest0-21swapper/210:51:562
392950899200,19cyclictest0-21swapper/210:07:422
392950899200,19cyclictest0-21swapper/209:43:492
392950899200,19cyclictest0-21swapper/209:31:122
392950899200,19cyclictest0-21swapper/209:08:072
392950899200,19cyclictest0-21swapper/208:27:382
392950899200,19cyclictest0-21swapper/207:40:542
392950899200,19cyclictest0-21swapper/207:24:252
3929506992014,4cyclictest2156-21gnome-shell12:21:091
3929506992012,6cyclictest3978623-21pluginstate09:25:201
392950699200,19cyclictest0-21swapper/110:02:411
392950699200,19cyclictest0-21swapper/107:41:541
3929501992018,1cyclictest0-21swapper/012:33:280
3929501992018,1cyclictest0-21swapper/012:05:360
3929501992018,1cyclictest0-21swapper/011:25:550
3929501992018,1cyclictest0-21swapper/010:12:280
3929501992018,1cyclictest0-21swapper/010:12:280
3929501992018,1cyclictest0-21swapper/009:52:270
3929501992018,1cyclictest0-21swapper/008:31:290
3929501992018,1cyclictest0-21swapper/007:55:460
3929501992018,1cyclictest0-21swapper/007:52:150
392950199200,19cyclictest0-21swapper/012:11:240
392950199200,19cyclictest0-21swapper/007:47:210
392950199200,19cyclictest0-21swapper/007:42:290
139099200,14rtkit-daemon0-21swapper/110:25:291
392951199199,8cyclictest42-21ksoftirqd/309:40:113
392951199194,13cyclictest3967266-21cat08:55:013
3929511991917,1cyclictest0-21swapper/310:07:573
3929511991914,3cyclictest3961106-21missed_timers08:35:173
3929511991913,5cyclictest4025423-21rm11:40:013
3929511991912,5cyclictest3981270-21sed09:35:013
3929511991912,5cyclictest3974114-21cron09:15:013
3929511991910,7cyclictest4041985-21df_inode12:25:133
392951199190,7cyclictest4008386-21/usr/sbin/munin10:50:163
392951199190,7cyclictest3967844-21forks08:55:153
392951199190,1cyclictest3056-21PK-Backend10:28:253
392951199190,1cyclictest3056-21PK-Backend09:07:273
392951199190,1cyclictest3056-21PK-Backend09:04:233
392951199190,1cyclictest0-21swapper/310:39:543
392951199190,1cyclictest0-21swapper/307:37:303
392951199190,18cyclictest3056-21PK-Backend09:37:353
392951199190,18cyclictest3056-21PK-Backend08:12:313
392951199190,18cyclictest0-21swapper/307:57:033
392951199190,18cyclictest0-21swapper/307:51:223
3929508991917,1cyclictest0-21swapper/207:37:222
3929508991916,2cyclictest3056-21PK-Backend12:22:142
3929508991914,4cyclictest34-21ksoftirqd/208:15:192
3929508991912,5cyclictest436-20systemd-journal11:40:292
392950899190,1cyclictest0-21swapper/210:31:282
392950899190,1cyclictest0-21swapper/210:20:532
392950899190,1cyclictest0-21swapper/209:03:072
3929506991917,1cyclictest0-21swapper/111:41:571
3929506991917,1cyclictest0-21swapper/110:31:001
3929506991912,5cyclictest3987106-21cut09:50:161
392950699191,16cyclictest4033243-21cat12:00:171
392950699190,1cyclictest0-21swapper/109:37:131
392950699190,1cyclictest0-21swapper/107:35:461
3929501991917,1cyclictest0-21swapper/009:42:260
3929501991917,1cyclictest0-21swapper/009:06:240
3929501991917,1cyclictest0-21swapper/008:27:440
3929501991917,1cyclictest0-21swapper/008:01:280
3929501991911,6cyclictest4029709-21ntpq11:50:190
392950199190,1cyclictest0-21swapper/012:21:380
392950199190,1cyclictest0-21swapper/010:00:240
392950199190,1cyclictest0-21swapper/009:49:360
392950199190,1cyclictest0-21swapper/008:23:580
392950199190,1cyclictest0-21swapper/008:23:580
392950199190,1cyclictest0-21swapper/008:10:280
139099190,13rtkit-daemon0-21swapper/008:06:520
40349042180,4sleep13056-21PK-Backend12:05:161
3929511991816,1cyclictest4002149-21systemd-run10:35:003
3929511991816,1cyclictest3056-21PK-Backend10:02:293
3929511991816,1cyclictest3056-21PK-Backend09:19:163
3929511991816,1cyclictest3056-21PK-Backend07:47:183
3929511991816,1cyclictest0-21swapper/309:45:193
3929511991813,3cyclictest4045469-21diskstats12:35:123
3929511991813,3cyclictest4009632-21apt10:55:113
3929511991811,5cyclictest3994234-21proc_pri10:10:213
3929511991811,5cyclictest3994234-21proc_pri10:10:203
3929508991816,1cyclictest0-21swapper/208:55:502
3929508991815,2cyclictest0-21swapper/207:52:162
3929508991813,3cyclictest978-21snmpd11:12:102
392950699189,7cyclictest3969651-21cat09:00:171
392950699183,8cyclictest141rcu_preempt08:05:021
392950699182,13cyclictest3960528-21apt08:35:121
3929506991816,1cyclictest0-21swapper/110:15:591
3929506991816,1cyclictest0-21swapper/110:15:591
3929506991815,2cyclictest3056-21PK-Backend09:42:131
3929506991814,2cyclictest3967182-21sh08:55:001
3929506991812,4cyclictest2156-21gnome-shell12:15:091
3929506991811,5cyclictest3997924-21ntp_states10:20:181
3929506991811,5cyclictest3985715-21cat09:45:201
3929506991810,6cyclictest3972369-21cron09:09:591
3929501991813,3cyclictest4016303-21dump-pmu-power11:15:000
3929501991811,6cyclictest4024965-21apt-get11:39:590
392950199181,15cyclictest4002108-21cron10:35:000
392950199180,1cyclictest0-21swapper/008:56:030
40163172170,1sleep14016319-21basename11:15:001
392951199179,6cyclictest3962519-21cat08:40:133
392951199174,2cyclictest4024513-21timerwakeupswit11:35:233
3929511991715,1cyclictest3056-21PK-Backend12:00:183
3929511991715,1cyclictest0-21swapper/308:21:103
3929511991715,1cyclictest0-21swapper/308:21:103
3929511991712,3cyclictest4036595-21if_enp2s012:10:153
3929511991710,5cyclictest4017900-21sshd11:18:293
392951199170,3cyclictest3932652-21awk07:15:173
392951199170,16cyclictest42-21ksoftirqd/311:05:103
392950899178,7cyclictest4012228-21python311:00:202
392950899178,5cyclictest4013277-21cpuspeed_turbos11:05:122
3929508991710,6cyclictest3989030-21pluginstate09:55:172
392950699179,6cyclictest436-20systemd-journal12:11:301
392950699179,6cyclictest3989118-21python309:55:171
3929506991715,1cyclictest0-21swapper/110:05:181
3929506991715,1cyclictest0-21swapper/109:13:541
3929506991715,1cyclictest0-21swapper/107:21:531
3929506991713,3cyclictest1095-21runrttasks08:25:291
3929506991712,3cyclictest3942867-21grep07:45:131
3929506991711,5cyclictest3976866-21ntpq09:20:181
3929506991710,5cyclictest3994263-21cut10:10:201
3929506991710,5cyclictest3994263-21cut10:10:201
3929501991715,1cyclictest0-21swapper/008:49:340
3929501991710,5cyclictest3977749-21sed09:25:000
40042872160,2sleep13056-21PK-Backend10:40:111
39401002160,3chrt0-21swapper/007:35:200
392951199169,6cyclictest4030460-21systemd11:55:013
3929511991611,3cyclictest4015578-21ntp_states11:10:193
3929511991610,4cyclictest42-21ksoftirqd/311:45:123
392950899162,13cyclictest3990656-21grep10:00:172
3929508991614,1cyclictest3056-21PK-Backend09:25:242
3929508991614,1cyclictest0-21swapper/210:40:342
3929508991611,4cyclictest978-21snmpd09:14:582
392950699169,6cyclictest4029303-21ls11:50:121
392950699169,5cyclictest3979689-21cat09:30:101
392950699169,5cyclictest1-21systemd11:50:011
392950699163,9cyclictest3935422-21apt07:25:121
392950699163,7cyclictest3951327-21apt-get08:10:011
3929506991611,4cyclictest3056-21PK-Backend10:56:031
392950699161,11cyclictest3974385-21apt09:15:121
3929506991610,5cyclictest3937778-21cat07:30:181
392950699160,6cyclictest3968914-21turbostat.cron09:00:001
3929501991614,1cyclictest0-21swapper/012:01:440
3929501991612,3cyclictest4009374-21idleruntime-cro10:55:000
392950199161,13cyclictest4004766-21needreboot10:40:160
3929511991510,4cyclictest3056-21PK-Backend09:56:303
392950899159,5cyclictest4009996-21df10:55:132
392950699156,7cyclictest3958945-21grep08:30:131
3929506991512,2cyclictest3056-21PK-Backend11:19:201
392950699151,11cyclictest4002389-21apt10:35:121
3929506991510,4cyclictest3056-21PK-Backend11:07:341
3929506991510,3cyclictest3056-21PK-Backend10:51:541
392950699150,14cyclictest0-21swapper/107:57:491
392950199153,4cyclictest13-21ksoftirqd/011:23:290
3929501991513,1cyclictest0-21swapper/011:19:260
3929501991510,4cyclictest4011237-21sed11:00:010
392951199149,4cyclictest3056-21PK-Backend07:25:093
392950699149,4cyclictest4023946-21awk11:35:171
392950699142,8cyclictest4041618-21apt12:25:101
3929506991412,1cyclictest3056-21PK-Backend11:00:201
3929506991412,1cyclictest3056-21PK-Backend08:18:581
392950699141,11cyclictest2156-21gnome-shell11:31:091
392950699140,1cyclictest3056-21PK-Backend07:51:411
392950699140,13cyclictest3056-21PK-Backend11:25:081
392950699140,13cyclictest0-21swapper/108:22:501
392950699140,13cyclictest0-21swapper/108:22:501
392950199148,4cyclictest4006224-21cpuspeed_turbos10:45:110
392950699139,3cyclictest2019-21gmain08:45:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional