You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-08 - 03:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Wed Apr 08, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
321213521250,2sleep10-21swapper/100:15:161
31670752950,1sleep00-21swapper/022:05:230
310283627739,13sleep20-21swapper/219:05:522
310283627739,13sleep20-21swapper/219:05:522
310292226444,13sleep30-21swapper/319:06:583
310292226444,13sleep30-21swapper/319:06:573
310302326344,13sleep00-21swapper/019:08:160
310302326344,13sleep00-21swapper/019:08:160
310204526343,13sleep10-21swapper/119:05:141
310204526343,13sleep10-21swapper/119:05:141
31648262360,4sleep20-21swapper/222:00:192
31160242350,1sleep33116028-21sed19:40:213
31201902340,3sleep20-21swapper/219:55:002
31579472310,5sleep30-21swapper/321:40:243
31352632300,4chrt3056-21PK-Backend20:35:211
31853042290,3sleep13056-21PK-Backend23:00:121
139099280,4rtkit-daemon1389-21rtkit-daemon20:42:122
310329199277,12cyclictest13-21ksoftirqd/021:05:000
139099270,4rtkit-daemon1389-21rtkit-daemon22:31:102
3103299992615,8cyclictest3162281-21seq21:55:032
139099260,4rtkit-daemon1389-21rtkit-daemon23:48:003
139099260,4rtkit-daemon1389-21rtkit-daemon22:56:572
3103302992515,7cyclictest436-20systemd-journal22:16:343
3103291992520,4cyclictest13-21ksoftirqd/021:10:010
3103291992516,2cyclictest13-21ksoftirqd/000:04:570
139099250,4rtkit-daemon1389-21rtkit-daemon20:03:073
31912862240,4sleep03056-21PK-Backend23:15:150
310330299247,15cyclictest3171663-21smartctl22:20:153
310329999240,23cyclictest1389-21rtkit-daemon22:51:322
3103291992416,6cyclictest3190306-21cron23:15:000
139099240,3rtkit-daemon1389-21rtkit-daemon20:43:553
3103302992314,7cyclictest3219934-21users00:35:243
3103295992315,6cyclictest3112696-21apt-get19:34:591
310329199234,2cyclictest141rcu_preempt23:35:090
3103291992320,2cyclictest3056-21PK-Backend22:30:040
3103302992220,1cyclictest3056-21PK-Backend23:57:453
3103302992220,1cyclictest3056-21PK-Backend21:35:153
3103302992220,1cyclictest3056-21PK-Backend00:22:113
3103302992220,1cyclictest0-21swapper/322:43:233
3103302992220,1cyclictest0-21swapper/300:11:373
3103302992219,2cyclictest3056-21PK-Backend20:28:163
3103302992219,2cyclictest0-21swapper/323:27:503
3103302992213,7cyclictest3161782-21apt-get21:55:013
310330299220,20cyclictest0-21swapper/320:55:073
3103299992215,5cyclictest3188573-21cron23:10:002
3103299992214,6cyclictest3194532-21seq23:25:142
3103295992216,4cyclictest3166389-21expr22:05:121
3103295992216,4cyclictest3107197-21chrt19:19:381
310329199223,9cyclictest13-21ksoftirqd/022:25:000
3103291992220,1cyclictest3056-21PK-Backend00:26:240
3103291992220,1cyclictest0-21swapper/023:50:340
3103291992220,1cyclictest0-21swapper/023:31:290
3103291992220,1cyclictest0-21swapper/023:31:280
3103291992218,3cyclictest13-21ksoftirqd/020:52:320
310329199220,9cyclictest3154857-21perf21:35:000
310329199220,9cyclictest3154857-21perf21:35:000
3103302992119,1cyclictest0-21swapper/323:53:523
3103302992119,1cyclictest0-21swapper/323:18:173
3103302992119,1cyclictest0-21swapper/321:23:053
3103302992119,1cyclictest0-21swapper/321:18:193
3103302992119,1cyclictest0-21swapper/300:32:463
3103302992119,1cyclictest0-21swapper/300:32:463
3103302992119,1cyclictest0-21swapper/300:04:403
3103302992118,2cyclictest3056-21PK-Backend22:14:443
3103302992114,5cyclictest3144448-21cron21:05:003
310330299210,20cyclictest3056-21PK-Backend22:03:273
310330299210,20cyclictest3056-21PK-Backend20:20:463
310330299210,20cyclictest3056-21PK-Backend00:05:423
310330299210,19cyclictest3056-21PK-Backend19:11:183
3103299992119,1cyclictest0-21swapper/219:30:462
3103299992116,3cyclictest3160053-21sh21:50:012
310329999210,19cyclictest0-21swapper/219:47:582
3103295992118,2cyclictest3056-21PK-Backend23:40:231
310329599210,20cyclictest0-21swapper/120:29:551
310329599210,20cyclictest0-21swapper/119:28:381
310329599210,1cyclictest3056-21PK-Backend22:41:491
310329599210,19cyclictest3056-21PK-Backend23:33:411
310329599210,19cyclictest3056-21PK-Backend23:33:411
310329599210,19cyclictest3056-21PK-Backend23:24:071
3103291992119,1cyclictest3056-21PK-Backend22:15:230
3103291992119,1cyclictest0-21swapper/023:49:330
3103291992119,1cyclictest0-21swapper/023:43:520
3103291992119,1cyclictest0-21swapper/022:42:480
3103291992119,1cyclictest0-21swapper/021:39:560
3103291992119,1cyclictest0-21swapper/021:19:470
3103291992119,1cyclictest0-21swapper/020:25:390
3103291992119,1cyclictest0-21swapper/019:58:160
3103291992119,1cyclictest0-21swapper/019:32:200
3103291992119,1cyclictest0-21swapper/000:34:550
3103291992119,1cyclictest0-21swapper/000:34:550
3103291992113,6cyclictest3160042-21cron21:50:000
3103291992113,6cyclictest3128435-21uptime20:15:220
3103291992112,6cyclictest3214862-21apt-get00:25:000
3103291992111,2cyclictest141rcu_preempt00:37:020
310329199210,20cyclictest0-21swapper/023:20:530
310329199210,20cyclictest0-21swapper/020:31:010
310329199210,20cyclictest0-21swapper/019:51:390
139099210,14rtkit-daemon3056-21PK-Backend00:32:381
139099210,14rtkit-daemon3056-21PK-Backend00:32:381
310330299206,9cyclictest883-21dbus-daemon20:40:013
310330299202,16cyclictest1-21systemd23:05:003
3103302992018,1cyclictest0-21swapper/322:51:543
3103302992018,1cyclictest0-21swapper/300:28:543
3103302992017,2cyclictest3056-21PK-Backend00:16:303
3103302992014,4cyclictest3189515-21grep23:10:173
3103302992012,6cyclictest3147920-21grep21:15:003
3103302992011,7cyclictest436-20systemd-journal23:40:003
310330299201,17cyclictest3173361-21df22:25:143
310330299201,16cyclictest3137786-21apt20:45:113
310330299200,1cyclictest3056-21PK-Backend22:06:043
310330299200,1cyclictest3056-21PK-Backend19:55:313
310330299200,1cyclictest3056-21PK-Backend19:27:123
310330299200,19cyclictest3056-21PK-Backend21:45:503
310330299200,19cyclictest3056-21PK-Backend19:32:133
310330299200,19cyclictest0-21swapper/323:34:333
310330299200,19cyclictest0-21swapper/323:34:323
310330299200,19cyclictest0-21swapper/323:06:083
310329999209,9cyclictest3154865-21cron21:35:002
310329999209,9cyclictest3154865-21cron21:35:002
3103299992018,1cyclictest0-21swapper/221:01:042
3103299992018,1cyclictest0-21swapper/220:27:242
3103299992018,1cyclictest0-21swapper/219:21:342
3103299992018,1cyclictest0-21swapper/200:37:212
3103299992018,1cyclictest0-21swapper/200:19:372
3103299992013,5cyclictest3181503-21cut22:50:002
3103299992012,6cyclictest3193318-21ntp_states23:20:182
310329999200,19cyclictest0-21swapper/223:54:292
310329999200,19cyclictest0-21swapper/222:17:162
310329999200,19cyclictest0-21swapper/222:08:232
310329999200,19cyclictest0-21swapper/221:28:282
310329999200,19cyclictest0-21swapper/221:07:122
310329999200,19cyclictest0-21swapper/220:50:302
310329999200,19cyclictest0-21swapper/220:21:162
310329999200,19cyclictest0-21swapper/220:00:342
310329999200,19cyclictest0-21swapper/200:26:332
310329999200,19cyclictest0-21swapper/200:09:372
3103295992018,1cyclictest3056-21PK-Backend23:52:331
3103295992018,1cyclictest0-21swapper/100:13:561
3103295992017,2cyclictest3056-21PK-Backend22:53:461
3103295992017,2cyclictest3056-21PK-Backend22:18:031
3103295992017,2cyclictest3056-21PK-Backend21:00:311
3103295992017,2cyclictest0-21swapper/121:40:111
310329599200,1cyclictest3056-21PK-Backend22:38:111
310329599200,1cyclictest3056-21PK-Backend20:04:331
310329599200,1cyclictest0-21swapper/123:57:131
310329599200,1cyclictest0-21swapper/119:54:341
310329599200,19cyclictest3056-21PK-Backend23:14:001
310329599200,19cyclictest3056-21PK-Backend20:52:321
310329599200,19cyclictest3056-21PK-Backend00:37:561
310329599200,19cyclictest0-21swapper/100:08:011
310329199204,7cyclictest13-21ksoftirqd/022:40:010
310329199204,4cyclictest13-21ksoftirqd/019:25:200
310329199202,10cyclictest13-21ksoftirqd/023:00:480
3103291992018,1cyclictest3056-21PK-Backend21:55:290
3103291992018,1cyclictest0-21swapper/023:56:160
3103291992018,1cyclictest0-21swapper/022:52:070
3103291992018,1cyclictest0-21swapper/021:12:180
3103291992018,1cyclictest0-21swapper/020:41:070
3103291992018,1cyclictest0-21swapper/020:38:450
3103291992018,1cyclictest0-21swapper/020:05:520
3103291992018,1cyclictest0-21swapper/000:11:570
3103291992018,1cyclictest0-21swapper/000:06:020
3103291992015,4cyclictest13-21ksoftirqd/019:50:000
310329199200,19cyclictest0-21swapper/023:09:450
310329199200,19cyclictest0-21swapper/022:56:470
310329199200,19cyclictest0-21swapper/022:01:310
310329199200,19cyclictest0-21swapper/020:48:510
310329199200,19cyclictest0-21swapper/020:03:160
31536922190,4sleep33056-21PK-Backend21:30:123
31536922190,4sleep33056-21PK-Backend21:30:123
31083752190,4chrt3056-21PK-Backend19:20:203
310330299193,14cyclictest0-21swapper/321:05:143
3103302991917,1cyclictest3056-21PK-Backend19:35:383
3103302991917,1cyclictest0-21swapper/323:40:273
3103302991917,1cyclictest0-21swapper/322:48:023
3103302991913,4cyclictest3128854-21idleruntime-cro20:20:003
3103302991911,6cyclictest3177169-21ntpq22:35:173
310330299190,1cyclictest3056-21PK-Backend20:11:543
310330299190,15cyclictest3056-21PK-Backend20:05:213
3103299991917,1cyclictest0-21swapper/222:26:072
3103299991917,1cyclictest0-21swapper/220:17:322
3103299991917,1cyclictest0-21swapper/200:21:392
3103299991912,5cyclictest3196567-21expr23:30:152
3103299991912,5cyclictest3196567-21expr23:30:152
3103299991911,6cyclictest3178479-21expr22:40:122
310329999190,1cyclictest3056-21PK-Backend19:28:092
310329999190,1cyclictest0-21swapper/223:40:022
310329999190,1cyclictest0-21swapper/223:04:272
310329999190,1cyclictest0-21swapper/222:10:122
310329999190,18cyclictest0-21swapper/223:35:092
310329999190,18cyclictest0-21swapper/222:21:412
3103295991917,1cyclictest3056-21PK-Backend22:49:061
3103295991917,1cyclictest0-21swapper/123:45:041
3103295991917,1cyclictest0-21swapper/123:15:021
3103295991917,1cyclictest0-21swapper/120:31:301
3103295991917,1cyclictest0-21swapper/120:14:061
3103295991917,1cyclictest0-21swapper/119:59:331
3103295991917,1cyclictest0-21swapper/100:22:411
3103295991916,2cyclictest3056-21PK-Backend23:25:101
3103295991916,2cyclictest3056-21PK-Backend22:27:311
3103295991912,5cyclictest3104663-21sshd19:10:411
3103295991911,6cyclictest3164424-21df22:00:131
310329599190,1cyclictest3056-21PK-Backend22:24:531
310329599190,1cyclictest3056-21PK-Backend21:13:551
310329599190,1cyclictest0-21swapper/120:47:461
310329599190,18cyclictest0-21swapper/122:12:021
310329599190,18cyclictest0-21swapper/100:03:081
310329199199,4cyclictest13-21ksoftirqd/021:21:000
310329199199,2cyclictest141rcu_preempt22:27:420
310329199198,3cyclictest13-21ksoftirqd/021:42:450
310329199197,4cyclictest13-21ksoftirqd/023:26:470
310329199197,4cyclictest13-21ksoftirqd/021:25:450
3103291991917,1cyclictest0-21swapper/020:10:050
3103291991917,1cyclictest0-21swapper/019:35:440
3103291991917,1cyclictest0-21swapper/019:12:120
3103291991917,1cyclictest0-21swapper/000:16:500
3103291991911,7cyclictest3107901-21awk19:20:140
310329199190,1cyclictest0-21swapper/022:46:260
310329199190,1cyclictest0-21swapper/022:13:340
310329199190,1cyclictest0-21swapper/020:24:040
310330299189,4cyclictest3152723-21cat21:25:213
3103302991816,1cyclictest0-21swapper/323:21:073
3103302991816,1cyclictest0-21swapper/322:31:263
3103302991812,4cyclictest42-21ksoftirqd/320:50:123
3103302991811,5cyclictest3161862-21apt-key21:55:003
310330299180,5cyclictest3183494-21sort22:55:123
3103299991816,1cyclictest3056-21PK-Backend20:30:352
3103299991816,1cyclictest0-21swapper/223:56:322
3103299991816,1cyclictest0-21swapper/220:10:492
3103299991816,1cyclictest0-21swapper/219:10:252
3103299991816,1cyclictest0-21swapper/200:32:292
3103299991816,1cyclictest0-21swapper/200:32:282
3103299991816,1cyclictest0-21swapper/200:12:402
3103299991811,5cyclictest3149908-21cat21:20:112
3103295991816,1cyclictest3056-21PK-Backend20:56:311
3103295991816,1cyclictest0-21swapper/122:30:071
3103295991813,3cyclictest3113678-21if_enp2s019:35:151
3103295991811,5cyclictest3117108-21which19:45:141
3103295991811,5cyclictest3109175-21sed19:25:021
3103295991810,6cyclictest883-21dbus-daemon20:17:311
310329199186,4cyclictest13-21ksoftirqd/020:58:380
3103291991815,2cyclictest13-21ksoftirqd/019:15:150
139099180,14rtkit-daemon3191589-21ntp_states23:15:192
3103302991715,1cyclictest0-21swapper/320:30:403
310329999178,7cyclictest3138151-21df_inode20:45:142
310329999172,4cyclictest34-21ksoftirqd/219:20:012
3103299991715,1cyclictest0-21swapper/221:39:162
3103299991715,1cyclictest0-21swapper/220:06:352
3103299991710,5cyclictest436-20systemd-journal20:00:002
3103299991710,5cyclictest3201454-21cat23:45:122
3103299991710,5cyclictest3115596-21cat19:40:182
310329599179,6cyclictest3158852-21expr21:45:121
3103295991715,1cyclictest3056-21PK-Backend23:37:331
3103295991715,1cyclictest3056-21PK-Backend22:56:371
3103295991715,1cyclictest3056-21PK-Backend21:05:511
3103295991715,1cyclictest0-21swapper/120:24:351
3103295991712,4cyclictest3155793-21strings21:35:171
3103295991712,4cyclictest26-21ksoftirqd/121:30:001
3103295991712,3cyclictest3161794-21latency_hist21:55:011
3103295991711,4cyclictest3115132-21grep19:40:121
3103295991710,5cyclictest3162873-21sed21:55:161
310329199177,3cyclictest13-21ksoftirqd/019:40:090
310329199176,3cyclictest141rcu_preempt21:50:080
310330299160,6cyclictest3056-21PK-Backend19:50:003
310330299160,2cyclictest31195812sleep319:50:203
310329999169,6cyclictest3135061-21ntp_states20:35:182
310329999169,5cyclictest436-20systemd-journal23:15:002
3103299991614,1cyclictest0-21swapper/221:19:292
3103299991611,3cyclictest3161790-21idleruntime-cro21:55:012
3103299991611,3cyclictest3147925-21munin-plugin-st21:15:002
3103299991610,5cyclictest3176938-21awk22:35:152
3103295991614,1cyclictest3056-21PK-Backend20:40:021
3103295991614,1cyclictest3056-21PK-Backend00:29:241
3103295991610,4cyclictest3150604-21cut21:20:171
31072642150,3sleep33056-21PK-Backend19:20:003
310329999158,6cyclictest3157410-21cat21:40:162
310329999158,5cyclictest436-20systemd-journal00:05:002
3103299991511,3cyclictest34-21ksoftirqd/220:55:082
3103299991511,3cyclictest3114691-21sh19:40:002
310329599159,4cyclictest1095-21runrttasks21:30:151
310329599159,4cyclictest1095-21runrttasks21:30:141
3103295991510,3cyclictest1095-21runrttasks20:05:011
310329599150,6cyclictest3056-21PK-Backend23:10:001
3103295991311,1cyclictest3056-21PK-Backend21:15:311
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional