You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-23 - 23:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Thu Jul 23, 2026 12:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
64882721160,2sleep33056-21PK-Backend10:55:463
56631921000,6sleep10-21swapper/107:05:101
139099890,54rtkit-daemon0-21swapper/007:09:050
56773826746,14sleep30-21swapper/307:09:243
56751026642,17sleep20-21swapper/207:06:242
6218352530,3sleep00-21swapper/009:40:170
5965122460,3sleep13056-21PK-Backend08:30:001
6043072450,4sleep33056-21PK-Backend08:50:163
6740162440,2sleep30-21swapper/312:10:143
6740162440,2sleep30-21swapper/312:10:143
5845812370,3sleep33056-21PK-Backend07:55:143
139099270,5rtkit-daemon632669-21awk10:10:230
139099260,17rtkit-daemon0-21swapper/010:55:200
139099250,16rtkit-daemon3056-21PK-Backend08:35:101
567943992411,4cyclictest34-21ksoftirqd/208:43:412
567943992411,4cyclictest34-21ksoftirqd/207:50:562
567947992319,3cyclictest641844-21users10:35:223
567943992310,4cyclictest34-21ksoftirqd/208:58:022
567935992318,3cyclictest664638-21users11:40:240
139099230,3rtkit-daemon0-21swapper/110:40:161
56794799228,13cyclictest42-21ksoftirqd/307:15:133
567947992220,1cyclictest3056-21PK-Backend11:51:593
567947992220,1cyclictest0-21swapper/309:33:083
567947992219,2cyclictest3056-21PK-Backend11:15:443
567947992219,2cyclictest3056-21PK-Backend09:47:223
56794799220,20cyclictest3056-21PK-Backend11:59:163
56794799220,20cyclictest3056-21PK-Backend09:06:463
56794799220,20cyclictest3056-21PK-Backend08:59:303
56794799220,20cyclictest3056-21PK-Backend08:48:493
56794399229,5cyclictest34-21ksoftirqd/209:04:482
567943992217,4cyclictest34-21ksoftirqd/208:25:012
567943992214,6cyclictest577002-21cron07:34:592
567943992214,5cyclictest589198-21ls08:10:002
567943992211,4cyclictest34-21ksoftirqd/209:40:392
567935992216,4cyclictest606460-21/usr/sbin/munin08:55:230
567947992119,1cyclictest0-21swapper/312:02:073
567947992119,1cyclictest0-21swapper/310:20:273
567947992119,1cyclictest0-21swapper/310:00:263
567947992119,1cyclictest0-21swapper/308:26:383
567947992119,1cyclictest0-21swapper/307:45:543
567947992118,2cyclictest3056-21PK-Backend11:22:263
567947992118,2cyclictest3056-21PK-Backend09:25:393
567947992118,2cyclictest3056-21PK-Backend09:18:573
567947992118,2cyclictest3056-21PK-Backend07:53:183
56794799210,7cyclictest598662-21apt08:35:123
56794799210,20cyclictest3056-21PK-Backend11:06:573
56794799210,20cyclictest0-21swapper/307:35:213
56794799210,19cyclictest3056-21PK-Backend10:43:193
56794799210,19cyclictest3056-21PK-Backend09:10:253
56794799210,19cyclictest3056-21PK-Backend09:10:243
56794399219,11cyclictest680539-21pool-evolution-12:28:492
56794399219,10cyclictest573582-21unin-run07:25:002
567943992114,6cyclictest646996-21cat10:50:222
567943992114,5cyclictest978-21snmpd10:35:152
567943992114,4cyclictest596394-21latency_hist08:30:012
567943992110,4cyclictest34-21ksoftirqd/209:45:252
567942992113,6cyclictest607052-21cron09:00:001
567942992113,6cyclictest1-21systemd09:30:001
56794299210,20cyclictest0-21swapper/110:30:151
567935992117,3cyclictest627787-21runrttasks09:59:240
567935992113,6cyclictest436-20systemd-journal09:55:010
56793599210,20cyclictest0-21swapper/012:31:210
56793599210,20cyclictest0-21swapper/008:54:220
56793599210,20cyclictest0-21swapper/007:39:530
139099210,14rtkit-daemon0-21swapper/011:35:070
56794799207,3cyclictest141rcu_preempt11:50:013
56794799205,3cyclictest569621-21systemctl07:15:003
567947992018,1cyclictest3056-21PK-Backend10:07:223
567947992018,1cyclictest3056-21PK-Backend10:07:223
567947992018,1cyclictest3056-21PK-Backend09:00:303
567947992018,1cyclictest0-21swapper/312:07:343
567947992018,1cyclictest0-21swapper/311:14:423
567947992018,1cyclictest0-21swapper/308:22:123
567947992018,1cyclictest0-21swapper/308:06:443
567947992018,1cyclictest0-21swapper/308:04:073
567947992018,1cyclictest0-21swapper/307:30:343
567947992015,3cyclictest592657-21cron08:20:003
567947992013,6cyclictest1-21systemd07:24:593
56794799201,17cyclictest597655-21ntpq08:30:183
56794799200,19cyclictest3056-21PK-Backend11:38:143
56794799200,19cyclictest3056-21PK-Backend10:27:233
56794799200,19cyclictest0-21swapper/312:31:353
56794799200,19cyclictest0-21swapper/312:20:523
56794799200,19cyclictest0-21swapper/307:25:473
56794399209,4cyclictest34-21ksoftirqd/211:33:082
56794399209,4cyclictest34-21ksoftirqd/211:28:352
56794399209,4cyclictest34-21ksoftirqd/209:39:322
56794399209,4cyclictest34-21ksoftirqd/209:12:162
56794399209,4cyclictest34-21ksoftirqd/209:12:162
56794399209,4cyclictest34-21ksoftirqd/208:48:062
56794399209,4cyclictest34-21ksoftirqd/207:46:402
56794399207,4cyclictest34-21ksoftirqd/210:00:172
56794399203,16cyclictest1095-21runrttasks08:02:332
56794399200,4cyclictest0-21swapper/211:37:532
56794299202,14cyclictest663567-21apt11:40:121
567942992018,1cyclictest0-21swapper/111:54:091
567942992017,2cyclictest3056-21PK-Backend07:15:271
567942992015,4cyclictest631317-21perf10:10:001
567942992015,4cyclictest631317-21perf10:10:001
567942992012,6cyclictest637788-21ntp_states10:25:201
567942992011,7cyclictest682382-21cron12:35:001
56794299200,1cyclictest3056-21PK-Backend12:23:301
56794299200,19cyclictest3056-21PK-Backend12:17:551
56794299200,19cyclictest3056-21PK-Backend08:41:591
56794299200,19cyclictest0-21swapper/110:55:571
567935992018,1cyclictest0-21swapper/012:29:300
567935992018,1cyclictest0-21swapper/012:16:120
567935992018,1cyclictest0-21swapper/011:58:000
567935992018,1cyclictest0-21swapper/011:12:320
567935992018,1cyclictest0-21swapper/010:45:490
567935992018,1cyclictest0-21swapper/009:26:120
567935992018,1cyclictest0-21swapper/008:35:500
56793599200,1cyclictest0-21swapper/010:21:410
56793599200,1cyclictest0-21swapper/009:03:270
56793599200,19cyclictest0-21swapper/012:37:360
56793599200,19cyclictest0-21swapper/011:48:420
56793599200,19cyclictest0-21swapper/011:27:110
56793599200,19cyclictest0-21swapper/010:02:280
56793599200,19cyclictest0-21swapper/008:49:560
56793599200,19cyclictest0-21swapper/007:29:320
56793599200,19cyclictest0-21swapper/007:20:260
56793599200,19cyclictest0-21swapper/007:17:280
56793599200,18cyclictest624400-21basename09:50:010
139099200,14rtkit-daemon0-21swapper/009:32:050
139099200,13rtkit-daemon0-21swapper/010:05:070
139099200,13rtkit-daemon0-21swapper/010:05:060
6798742190,2sleep30-21swapper/312:25:163
6639682190,4sleep234-21ksoftirqd/211:40:162
567947991917,1cyclictest3056-21PK-Backend11:43:423
567947991917,1cyclictest3056-21PK-Backend11:26:173
567947991917,1cyclictest3056-21PK-Backend10:47:253
567947991917,1cyclictest3056-21PK-Backend09:50:273
567947991917,1cyclictest0-21swapper/312:39:393
567947991917,1cyclictest0-21swapper/310:34:193
567947991916,2cyclictest0-21swapper/311:00:153
567947991913,5cyclictest600736-21ls08:40:133
567947991912,6cyclictest589442-21sshd08:10:083
567947991912,6cyclictest589442-21sshd08:10:083
56794799190,1cyclictest3056-21PK-Backend09:43:293
56794799190,1cyclictest0-21swapper/312:16:263
56794799190,1cyclictest0-21swapper/310:16:223
56794799190,1cyclictest0-21swapper/310:13:183
56794399199,3cyclictest34-21ksoftirqd/211:05:412
56794399198,4cyclictest34-21ksoftirqd/211:54:222
56794399198,4cyclictest34-21ksoftirqd/211:46:512
56794399198,4cyclictest34-21ksoftirqd/210:43:072
56794399198,4cyclictest34-21ksoftirqd/210:09:362
56794399198,4cyclictest34-21ksoftirqd/210:09:352
56794399198,3cyclictest34-21ksoftirqd/210:48:532
56794399197,4cyclictest34-21ksoftirqd/209:28:542
56794399197,3cyclictest34-21ksoftirqd/209:07:402
567943991914,4cyclictest34-21ksoftirqd/208:15:012
567943991914,4cyclictest34-21ksoftirqd/208:15:002
567943991912,5cyclictest626933-21cat09:55:152
567943991912,5cyclictest585192-21cat07:55:222
567943991911,6cyclictest883-21dbus-daemon10:20:012
567943991911,6cyclictest650177-21ntpq11:00:192
567943991911,5cyclictest682011-21sshd12:30:252
567943991910,7cyclictest883-21dbus-daemon12:04:592
56794399190,9cyclictest647758-21cpuspeed10:55:122
56794399190,2cyclictest141rcu_preempt09:21:562
567942991917,1cyclictest3056-21PK-Backend08:11:031
567942991917,1cyclictest3056-21PK-Backend08:11:031
567942991917,1cyclictest0-21swapper/111:45:311
567942991917,1cyclictest0-21swapper/110:39:551
567942991917,1cyclictest0-21swapper/109:24:441
567942991917,1cyclictest0-21swapper/107:59:481
567942991913,5cyclictest604606-21ntpq08:50:181
567942991912,5cyclictest669682-21cat11:55:211
567942991912,5cyclictest652674-21pluginstate11:10:011
567942991911,6cyclictest654626-21cron11:14:591
56794299190,1cyclictest3056-21PK-Backend12:14:441
56794299190,1cyclictest3056-21PK-Backend12:14:441
56794299190,1cyclictest3056-21PK-Backend08:06:301
56794299190,1cyclictest0-21swapper/111:33:151
56794299190,1cyclictest0-21swapper/111:15:161
56794299190,1cyclictest0-21swapper/110:48:251
56794299190,1cyclictest0-21swapper/107:46:431
56794299190,18cyclictest3056-21PK-Backend07:43:451
56794299190,18cyclictest0-21swapper/110:54:541
56794299190,18cyclictest0-21swapper/109:00:591
56793599198,9cyclictest957-21gdbus12:10:460
56793599198,9cyclictest957-21gdbus12:10:450
567935991917,1cyclictest0-21swapper/012:09:580
567935991917,1cyclictest0-21swapper/011:22:180
567935991917,1cyclictest0-21swapper/010:42:450
567935991917,1cyclictest0-21swapper/010:38:390
567935991917,1cyclictest0-21swapper/010:29:390
567935991917,1cyclictest0-21swapper/008:26:570
567935991917,1cyclictest0-21swapper/008:17:180
567935991917,1cyclictest0-21swapper/007:48:040
567935991911,6cyclictest649538-21cat11:00:120
56793599190,1cyclictest3056-21PK-Backend10:30:410
56793599190,1cyclictest0-21swapper/011:34:420
56793599190,1cyclictest0-21swapper/010:54:470
56793599190,1cyclictest0-21swapper/009:37:340
56793599190,1cyclictest0-21swapper/009:15:380
56793599190,1cyclictest0-21swapper/008:14:410
56793599190,1cyclictest0-21swapper/008:14:400
56793599190,1cyclictest0-21swapper/007:31:560
56793599190,18cyclictest0-21swapper/011:17:260
56793599190,18cyclictest0-21swapper/010:19:400
56793599190,18cyclictest0-21swapper/008:22:320
5972542180,3sleep13056-21PK-Backend08:30:131
567947991811,6cyclictest579811-21ntp_states07:40:193
56794399187,4cyclictest34-21ksoftirqd/210:27:242
56794399187,4cyclictest34-21ksoftirqd/207:27:572
56794399183,14cyclictest1095-21runrttasks12:19:332
567943991812,4cyclictest598386-21dump-pmu-power08:35:012
567943991812,4cyclictest34-21ksoftirqd/212:00:012
567943991811,5cyclictest655683-21which11:15:192
567943991811,5cyclictest568127-21awk07:10:112
567943991811,5cyclictest436-20systemd-journal08:16:592
567943991810,6cyclictest1-21systemd10:35:002
56794299189,7cyclictest628488-21df_inode10:00:151
567942991812,5cyclictest583572-21cat07:50:231
567942991812,3cyclictest26-21ksoftirqd/109:50:111
567942991811,5cyclictest657167-21cat11:20:161
567942991810,2cyclictest26-21ksoftirqd/110:25:001
567935991816,1cyclictest0-21swapper/011:07:390
567935991816,1cyclictest0-21swapper/008:31:230
567935991816,1cyclictest0-21swapper/008:09:400
567935991816,1cyclictest0-21swapper/007:59:410
56793599180,1cyclictest0-21swapper/009:08:550
567947991715,1cyclictest3056-21PK-Backend11:30:093
567947991715,1cyclictest3056-21PK-Backend09:59:263
567947991715,1cyclictest3056-21PK-Backend09:37:003
567947991710,5cyclictest646706-21ntpq10:50:183
56794399179,5cyclictest653894-21ntp_offset11:10:172
567943991715,1cyclictest0-21swapper/212:12:352
567943991715,1cyclictest0-21swapper/212:12:342
567943991715,1cyclictest0-21swapper/209:30:052
567943991715,1cyclictest0-21swapper/207:15:342
567943991712,3cyclictest34-21ksoftirqd/212:09:592
567943991711,5cyclictest577851-21cat07:35:172
567943991710,5cyclictest34-21ksoftirqd/207:40:202
56794399170,15cyclictest436-20systemd-journal12:24:262
56794299179,6cyclictest659499-21sshd11:25:581
56794299179,6cyclictest611514-21ntpq09:10:191
56794299179,6cyclictest611514-21ntpq09:10:191
56794299172,11cyclictest621191-21apt09:40:121
567942991715,1cyclictest4176944-21snapd09:36:141
567942991715,1cyclictest3056-21PK-Backend10:16:421
567942991715,1cyclictest3056-21PK-Backend07:31:021
567942991712,4cyclictest26-21ksoftirqd/108:45:111
567942991712,2cyclictest26-21ksoftirqd/109:05:121
567942991710,5cyclictest624569-21date09:50:011
56793599179,6cyclictest611718-21sed09:10:220
56793599179,6cyclictest611718-21sed09:10:220
567935991715,1cyclictest0-21swapper/009:20:320
567935991710,5cyclictest580132-21cat07:40:210
6570342160,3sleep2331rcuc/211:20:162
567947991614,1cyclictest3056-21PK-Backend09:22:493
567943991614,1cyclictest0-21swapper/208:35:362
567943991612,3cyclictest1095-21runrttasks12:36:042
56794399161,13cyclictest604022-21grep08:50:132
567942991614,1cyclictest3056-21PK-Backend12:05:261
567942991614,1cyclictest3056-21PK-Backend11:02:121
567942991614,1cyclictest0-21swapper/109:58:161
567942991612,2cyclictest26-21ksoftirqd/112:00:121
567942991612,2cyclictest26-21ksoftirqd/107:24:131
56794299161,11cyclictest661835-21apt11:35:131
567935991614,1cyclictest0-21swapper/008:04:410
567935991614,1cyclictest0-21swapper/007:54:400
567935991614,1cyclictest0-21swapper/007:11:480
567935991611,4cyclictest601064-21cut08:40:170
567935991611,3cyclictest671955-21sh12:04:590
567935991610,5cyclictest668459-21seq11:54:460
56794399158,5cyclictest635048-21apt10:20:102
56794399158,5cyclictest631877-21cat10:10:122
56794399154,2cyclictest141rcu_preempt09:15:222
56794399153,2cyclictest141rcu_preempt09:52:092
56794299157,6cyclictest631819-21cat10:10:121
567942991513,1cyclictest3056-21PK-Backend07:36:421
567942991513,1cyclictest3056-21PK-Backend07:11:341
567942991513,1cyclictest0-21swapper/109:18:281
567942991512,2cyclictest3056-21PK-Backend08:20:081
56794299151,10cyclictest586742-21ntp_states08:00:171
56794299149,3cyclictest3056-21PK-Backend12:35:131
567942991412,1cyclictest3056-21PK-Backend09:33:371
567942991412,1cyclictest3056-21PK-Backend07:28:521
56794299140,13cyclictest3056-21PK-Backend08:15:361
56793599148,5cyclictest677927-21cut12:20:160
56794299130,4cyclictest141rcu_preempt12:30:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional