You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-17 - 09:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackcslot7s.osadl.org (updated Fri Apr 17, 2026 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
406077821130,3sleep20-21swapper/220:25:112
139099930,58rtkit-daemon0-21swapper/019:06:560
403303327757,14sleep20-21swapper/219:09:092
403308627437,14sleep30-21swapper/319:09:523
403160526242,14sleep10-21swapper/119:05:021
40341022580,14sleep342-21ksoftirqd/319:10:153
41453032500,4sleep33056-21PK-Backend00:25:133
40903132490,1sleep20-21swapper/221:50:002
41142542480,2sleep00-21swapper/022:55:190
40356112460,4sleep23056-21PK-Backend19:15:032
41302832450,2sleep00-21swapper/023:40:220
41379142310,4sleep13056-21PK-Backend00:05:011
403332699291,26cyclictest4052334-21/usr/sbin/munin20:00:201
403334099276,3cyclictest42-21ksoftirqd/322:00:003
139099270,4rtkit-daemon1389-21rtkit-daemon22:50:462
139099260,4rtkit-daemon1389-21rtkit-daemon23:30:302
139099260,4rtkit-daemon1389-21rtkit-daemon23:30:302
403332699250,22cyclictest4118513-21munin-run23:10:001
403334099244,2cyclictest141rcu_preempt22:36:243
4033340992418,4cyclictest42-21ksoftirqd/300:15:013
4033340992415,3cyclictest42-21ksoftirqd/323:55:013
4033333992314,7cyclictest4095299-21users22:00:222
4033326992321,1cyclictest0-21swapper/122:43:101
4033326992320,2cyclictest3056-21PK-Backend22:05:381
4033323992320,2cyclictest0-21swapper/019:22:510
40456382220,3sleep3411rcuc/319:40:213
4033340992220,1cyclictest0-21swapper/323:31:053
4033340992220,1cyclictest0-21swapper/323:31:053
4033340992220,1cyclictest0-21swapper/300:30:133
403334099220,20cyclictest3056-21PK-Backend23:39:373
403334099220,20cyclictest3056-21PK-Backend19:30:143
403334099220,20cyclictest3056-21PK-Backend19:30:143
4033333992214,6cyclictest4086480-21apt-get21:40:002
4033326992220,1cyclictest0-21swapper/121:20:231
4033326992220,1cyclictest0-21swapper/100:23:071
4033326992216,4cyclictest4047363-21timerwakeupswit19:45:221
4033326992215,5cyclictest4037384-21cron19:20:001
4033326992213,7cyclictest4095086-21python322:00:211
403332699220,20cyclictest3056-21PK-Backend20:27:581
4033323992220,1cyclictest0-21swapper/023:28:490
4033323992220,1cyclictest0-21swapper/022:48:410
4033323992220,1cyclictest0-21swapper/020:17:400
4033323992220,1cyclictest0-21swapper/019:45:080
4033323992220,1cyclictest0-21swapper/019:38:050
403332399220,21cyclictest0-21swapper/022:23:050
403332399220,21cyclictest0-21swapper/020:42:130
41088622210,2sleep33056-21PK-Backend22:40:193
40696772210,5chrt0-21swapper/120:50:181
4033340992119,1cyclictest3056-21PK-Backend20:38:223
4033340992119,1cyclictest3056-21PK-Backend19:37:383
4033340992119,1cyclictest0-21swapper/323:45:313
4033340992119,1cyclictest0-21swapper/323:17:123
4033340992119,1cyclictest0-21swapper/322:09:463
4033340992119,1cyclictest0-21swapper/321:01:413
4033340992119,1cyclictest0-21swapper/320:15:023
4033340992118,2cyclictest3056-21PK-Backend21:21:203
4033340992118,2cyclictest3056-21PK-Backend21:18:433
403334099211,7cyclictest4058731-21apt20:20:113
4033340992116,4cyclictest42-21ksoftirqd/300:10:013
403334099210,20cyclictest0-21swapper/323:57:353
403334099210,20cyclictest0-21swapper/323:03:473
403334099210,1cyclictest0-21swapper/320:25:163
403334099210,19cyclictest3056-21PK-Backend22:30:213
403334099210,19cyclictest3056-21PK-Backend22:30:213
403334099210,19cyclictest3056-21PK-Backend19:48:263
4033333992119,1cyclictest0-21swapper/220:21:032
4033333992114,5cyclictest34-21ksoftirqd/223:15:112
4033333992110,9cyclictest1788610-21gdbus20:46:022
403333399210,20cyclictest0-21swapper/200:07:412
403333399210,1cyclictest0-21swapper/219:36:222
403333399210,19cyclictest0-21swapper/221:34:242
403333399210,19cyclictest0-21swapper/221:34:242
4033326992119,1cyclictest3056-21PK-Backend23:42:381
4033326992119,1cyclictest3056-21PK-Backend20:09:251
4033326992119,1cyclictest0-21swapper/123:36:441
4033326992119,1cyclictest0-21swapper/122:26:071
4033326992119,1cyclictest0-21swapper/122:21:271
4033326992119,1cyclictest0-21swapper/122:13:581
4033326992119,1cyclictest0-21swapper/121:38:071
4033326992119,1cyclictest0-21swapper/119:35:321
4033326992119,1cyclictest0-21swapper/119:32:561
4033326992119,1cyclictest0-21swapper/119:32:551
4033326992119,1cyclictest0-21swapper/100:26:121
4033326992112,7cyclictest4107506-21gdbus22:36:051
403332699210,1cyclictest3056-21PK-Backend00:14:561
403332699210,19cyclictest3056-21PK-Backend23:51:371
403332699210,19cyclictest3056-21PK-Backend23:16:561
4033323992119,1cyclictest0-21swapper/023:05:100
4033323992119,1cyclictest0-21swapper/023:04:080
4033323992119,1cyclictest0-21swapper/022:37:050
4033323992119,1cyclictest0-21swapper/022:29:340
4033323992119,1cyclictest0-21swapper/022:13:460
4033323992119,1cyclictest0-21swapper/020:56:340
4033323992119,1cyclictest0-21swapper/020:50:180
4033323992119,1cyclictest0-21swapper/020:28:080
4033323992119,1cyclictest0-21swapper/019:51:230
4033323992119,1cyclictest0-21swapper/019:43:190
4033323992119,1cyclictest0-21swapper/019:30:290
4033323992119,1cyclictest0-21swapper/019:30:280
4033323992119,1cyclictest0-21swapper/019:27:030
4033323992119,1cyclictest0-21swapper/000:39:330
4033323992118,2cyclictest3056-21PK-Backend19:55:020
4033323992113,6cyclictest4078174-21df21:15:130
403332399210,20cyclictest0-21swapper/021:56:560
403332399210,20cyclictest0-21swapper/021:49:260
403332399210,20cyclictest0-21swapper/021:42:570
403332399210,20cyclictest0-21swapper/021:36:420
403332399210,20cyclictest0-21swapper/021:30:260
403332399210,20cyclictest0-21swapper/021:30:260
403332399210,20cyclictest0-21swapper/021:22:220
403332399210,20cyclictest0-21swapper/020:06:450
139099210,3rtkit-daemon1389-21rtkit-daemon22:15:372
41191912200,3sleep10-21swapper/123:10:151
4033340992018,1cyclictest3056-21PK-Backend21:28:093
4033340992018,1cyclictest3056-21PK-Backend19:18:313
4033340992018,1cyclictest3056-21PK-Backend00:36:553
4033340992018,1cyclictest0-21swapper/322:01:283
4033340992018,1cyclictest0-21swapper/321:53:103
4033340992018,1cyclictest0-21swapper/320:32:193
4033340992018,1cyclictest0-21swapper/320:04:563
4033340992018,1cyclictest0-21swapper/300:04:323
4033340992017,2cyclictest3056-21PK-Backend22:55:163
4033340992017,2cyclictest3056-21PK-Backend21:38:373
4033340992017,2cyclictest3056-21PK-Backend00:20:403
403334099200,19cyclictest3056-21PK-Backend20:45:583
403334099200,19cyclictest0-21swapper/323:28:013
403334099200,19cyclictest0-21swapper/321:41:143
4033333992018,1cyclictest0-21swapper/223:40:512
4033333992018,1cyclictest0-21swapper/221:55:532
4033333992018,1cyclictest0-21swapper/219:11:482
4033333992018,1cyclictest0-21swapper/200:21:272
4033333992013,5cyclictest4053825-21cat20:05:152
403333399200,1cyclictest0-21swapper/223:09:212
403333399200,1cyclictest0-21swapper/222:35:552
403333399200,1cyclictest0-21swapper/219:40:492
403333399200,1cyclictest0-21swapper/219:21:422
403333399200,19cyclictest0-21swapper/223:56:122
403333399200,19cyclictest0-21swapper/223:36:452
403333399200,19cyclictest0-21swapper/223:11:242
403333399200,19cyclictest0-21swapper/222:34:542
403333399200,19cyclictest0-21swapper/222:34:542
403333399200,19cyclictest0-21swapper/222:26:222
403333399200,19cyclictest0-21swapper/222:10:062
403333399200,19cyclictest0-21swapper/221:42:422
403333399200,19cyclictest0-21swapper/221:11:472
403333399200,19cyclictest0-21swapper/221:00:512
403333399200,19cyclictest0-21swapper/220:34:212
403333399200,19cyclictest0-21swapper/219:55:212
4033326992018,1cyclictest3056-21PK-Backend23:30:491
4033326992018,1cyclictest3056-21PK-Backend23:30:491
4033326992018,1cyclictest3056-21PK-Backend23:28:461
4033326992018,1cyclictest0-21swapper/122:51:551
4033326992018,1cyclictest0-21swapper/122:18:361
4033326992018,1cyclictest0-21swapper/121:27:261
4033326992018,1cyclictest0-21swapper/120:48:191
4033326992018,1cyclictest0-21swapper/119:40:461
4033326992018,1cyclictest0-21swapper/100:19:021
4033326992017,2cyclictest3056-21PK-Backend23:55:431
4033326992015,4cyclictest26-21ksoftirqd/121:41:041
4033326992013,5cyclictest4149364-21cat00:35:121
403332699201,17cyclictest4083920-21grep21:30:171
403332699201,17cyclictest4083920-21grep21:30:171
403332699200,1cyclictest3056-21PK-Backend23:22:511
403332699200,19cyclictest3056-21PK-Backend21:50:511
403332699200,19cyclictest0-21swapper/122:49:531
403332699200,19cyclictest0-21swapper/122:30:471
403332699200,19cyclictest0-21swapper/122:30:461
403332699200,19cyclictest0-21swapper/120:22:451
403332699200,19cyclictest0-21swapper/119:58:301
403332699200,19cyclictest0-21swapper/119:25:191
403332699200,18cyclictest0-21swapper/120:35:011
4033323992018,1cyclictest0-21swapper/023:30:520
4033323992018,1cyclictest0-21swapper/023:30:520
4033323992018,1cyclictest0-21swapper/022:30:360
4033323992018,1cyclictest0-21swapper/022:30:360
4033323992018,1cyclictest0-21swapper/021:10:050
4033323992018,1cyclictest0-21swapper/020:22:530
4033323992018,1cyclictest0-21swapper/020:03:070
4033323992018,1cyclictest0-21swapper/019:13:450
4033323992018,1cyclictest0-21swapper/000:20:330
4033323992013,5cyclictest4119344-21load23:10:160
4033323992012,6cyclictest4082423-21ls21:25:190
403332399200,1cyclictest3056-21PK-Backend20:35:100
403332399200,1cyclictest0-21swapper/022:03:250
403332399200,19cyclictest0-21swapper/023:36:470
403332399200,19cyclictest0-21swapper/022:52:320
403332399200,19cyclictest0-21swapper/022:18:260
403332399200,19cyclictest0-21swapper/021:52:170
403332399200,19cyclictest0-21swapper/021:04:370
403332399200,19cyclictest0-21swapper/000:31:350
139099200,2rtkit-daemon1389-21rtkit-daemon20:09:243
4033340991917,1cyclictest3056-21PK-Backend23:40:383
4033340991917,1cyclictest3056-21PK-Backend23:11:183
4033340991917,1cyclictest3056-21PK-Backend21:06:063
4033340991917,1cyclictest0-21swapper/322:15:143
4033340991917,1cyclictest0-21swapper/320:53:353
4033340991917,1cyclictest0-21swapper/320:11:243
4033340991917,1cyclictest0-21swapper/319:54:273
4033340991916,2cyclictest3056-21PK-Backend22:48:473
4033340991913,5cyclictest42-21ksoftirqd/321:46:043
403334099190,1cyclictest3056-21PK-Backend20:42:333
403334099190,1cyclictest0-21swapper/320:55:243
4033333991917,1cyclictest0-21swapper/223:20:362
4033333991917,1cyclictest0-21swapper/222:48:462
4033333991917,1cyclictest0-21swapper/222:42:512
4033333991917,1cyclictest0-21swapper/222:20:412
4033333991917,1cyclictest0-21swapper/221:53:032
4033333991917,1cyclictest0-21swapper/220:54:222
4033333991917,1cyclictest0-21swapper/220:14:492
4033333991913,5cyclictest34-21ksoftirqd/223:25:112
4033333991913,5cyclictest1-21systemd00:12:192
4033333991912,5cyclictest4133346-21cat23:50:182
4033333991912,5cyclictest34-21ksoftirqd/223:00:122
403333399190,1cyclictest0-21swapper/222:55:422
403333399190,1cyclictest0-21swapper/219:31:082
403333399190,1cyclictest0-21swapper/219:31:082
403333399190,18cyclictest0-21swapper/221:27:072
403333399190,18cyclictest0-21swapper/221:23:292
403333399190,18cyclictest0-21swapper/221:15:242
403333399190,18cyclictest0-21swapper/220:38:002
403333399190,18cyclictest0-21swapper/200:26:342
4033326991917,1cyclictest3056-21PK-Backend21:47:131
4033326991917,1cyclictest3056-21PK-Backend21:02:391
4033326991917,1cyclictest3056-21PK-Backend20:10:271
4033326991916,2cyclictest3056-21PK-Backend21:55:311
4033326991915,3cyclictest0-21swapper/100:30:171
4033326991912,5cyclictest4049153-21sshd19:50:251
4033326991912,5cyclictest4034665-21sed19:10:201
403332699190,1cyclictest3056-21PK-Backend23:47:321
403332699190,1cyclictest3056-21PK-Backend23:01:291
403332699190,1cyclictest3056-21PK-Backend22:57:381
403332699190,1cyclictest3056-21PK-Backend20:59:001
403332699190,1cyclictest3056-21PK-Backend00:09:491
403332699190,18cyclictest0-21swapper/120:42:041
4033323991917,1cyclictest0-21swapper/022:44:490
4033323991917,1cyclictest0-21swapper/021:05:390
4033323991917,1cyclictest0-21swapper/020:11:240
4033323991917,1cyclictest0-21swapper/000:15:260
4033323991912,5cyclictest4145655-21expr00:25:140
4033323991912,5cyclictest4035483-21cat19:15:020
403332399190,1cyclictest0-21swapper/020:46:390
403332399190,18cyclictest0-21swapper/023:20:510
403332399190,18cyclictest0-21swapper/023:18:490
403334099189,2cyclictest141rcu_preempt21:32:533
403334099189,2cyclictest141rcu_preempt21:32:523
403334099187,3cyclictest42-21ksoftirqd/322:27:503
403334099186,11cyclictest42-21ksoftirqd/323:22:123
4033340991813,3cyclictest4117898-21python323:05:203
4033340991813,3cyclictest4038029-21diskstats19:20:133
403334099181,2cyclictest3056-21PK-Backend22:15:003
403333399182,12cyclictest4131167-21apt23:45:112
4033333991816,1cyclictest0-21swapper/222:07:162
4033333991816,1cyclictest0-21swapper/221:07:202
4033333991816,1cyclictest0-21swapper/220:01:502
4033333991811,5cyclictest34-21ksoftirqd/220:55:142
4033333991811,5cyclictest34-21ksoftirqd/200:15:122
4033326991816,1cyclictest3056-21PK-Backend20:32:241
4033326991816,1cyclictest0-21swapper/121:15:571
4033326991816,1cyclictest0-21swapper/121:14:081
403332699180,1cyclictest0-21swapper/121:07:051
4033323991811,5cyclictest4137284-21seq00:00:190
40488332170,2chrt0-21swapper/219:50:192
4033340991711,4cyclictest4112508-21cut22:50:183
403334099170,16cyclictest3056-21PK-Backend19:57:043
403333399176,2cyclictest141rcu_preempt20:40:132
4033333991715,1cyclictest0-21swapper/219:46:162
4033333991711,5cyclictest34-21ksoftirqd/220:15:112
4033323991715,1cyclictest0-21swapper/022:07:170
4033323991714,2cyclictest4134350-21cron23:55:000
4033340991610,4cyclictest42-21ksoftirqd/300:15:143
403334099160,3cyclictest4040710-21chrt19:28:223
403334099160,2cyclictest141rcu_preempt21:14:243
403333399169,5cyclictest4149220-21tr00:35:122
4033333991614,1cyclictest0-21swapper/219:27:432
4033326991614,1cyclictest0-21swapper/120:15:541
4033323991614,1cyclictest0-21swapper/000:11:210
4033323991610,4cyclictest4135076-21cat23:55:160
4033340991513,1cyclictest3056-21PK-Backend22:21:303
4033333991510,4cyclictest4136534-21tr00:00:102
403332699158,5cyclictest4038576-21cat19:20:211
403332399159,5cyclictest4131750-21wc23:45:160
4033323991513,1cyclictest0-21swapper/020:32:340
4033323991411,2cyclictest3056-21PK-Backend00:06:140
403333399136,5cyclictest4147731-21basename00:30:162
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional