You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-11 - 22:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sat Apr 11, 2026 12:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
97892621300,4sleep20-21swapper/210:30:222
95660821230,3sleep33056-21PK-Backend09:30:133
96368021090,5sleep33056-21PK-Backend09:50:123
99757421050,4sleep23056-21PK-Backend11:25:162
1390991030,56rtkit-daemon0-21swapper/007:09:590
10030202990,3sleep33056-21PK-Backend11:40:173
90483927947,22sleep20-21swapper/207:05:082
9961522780,3sleep33056-21PK-Backend11:20:203
9153692740,4sleep23056-21PK-Backend07:30:212
90612327039,24sleep30-21swapper/307:07:343
90599926444,13sleep10-21swapper/107:05:551
9473352520,4sleep10-21swapper/109:05:111
9739412480,4sleep33056-21PK-Backend10:20:003
9438712470,4sleep33056-21PK-Backend08:55:123
9428792460,3sleep00-21swapper/008:50:200
10097932390,2sleep10-21swapper/112:00:151
90644899338,8cyclictest955906-21chrt09:29:090
9341652290,4sleep33056-21PK-Backend08:25:173
90644899287,8cyclictest2156-21gnome-shell12:17:080
90644899274,8cyclictest921270-21cat07:50:010
90644899267,6cyclictest0-21swapper/007:18:080
90644899264,13cyclictest0-21swapper/012:00:110
90644899264,13cyclictest0-21swapper/009:50:130
90644899259,14cyclictest1016757-21uname12:20:170
90644899256,6cyclictest2156-21gnome-shell11:06:080
90644899254,8cyclictest0-21swapper/012:25:170
906448992514,7cyclictest989113-21seq11:00:230
906448992514,7cyclictest989113-21seq11:00:230
90644899248,14cyclictest971853-21perf10:14:590
90644899245,7cyclictest975063-21find10:20:210
90644899243,9cyclictest0-21swapper/011:10:200
90644899243,8cyclictest0-21swapper/009:35:030
906454992321,1cyclictest0-21swapper/209:56:262
906451992321,1cyclictest0-21swapper/107:54:411
906451992314,7cyclictest918-21rs:main1
906451992314,7cyclictest1013659-21python312:10:201
90644899233,8cyclictest0-21swapper/010:25:040
906448992320,2cyclictest913471-21python307:25:220
906448992314,6cyclictest917868-21apt07:40:120
906448992313,8cyclictest1005597-21awk11:50:000
90644899231,20cyclictest1005851-21apt11:50:120
906448992311,11cyclictest1000374-21dump-pmu-power11:35:000
906448992310,11cyclictest1022974-21unin-run12:40:010
10162112230,3chrt3056-21PK-Backend12:20:012
906458992220,1cyclictest3056-21PK-Backend10:25:463
906458992219,2cyclictest3056-21PK-Backend10:47:003
90645899220,20cyclictest0-21swapper/308:05:193
90645899220,20cyclictest0-21swapper/308:05:193
906454992220,1cyclictest0-21swapper/212:08:142
906454992220,1cyclictest0-21swapper/211:56:182
906454992220,1cyclictest0-21swapper/211:56:182
906454992220,1cyclictest0-21swapper/211:53:282
906454992220,1cyclictest0-21swapper/210:35:342
90645499220,20cyclictest0-21swapper/207:46:082
906451992213,7cyclictest1003638-21python311:42:351
906448992220,1cyclictest0-21swapper/011:26:460
906448992218,3cyclictest0-21swapper/009:45:060
906448992212,8cyclictest985309-21sed10:50:170
906448992212,8cyclictest982184-21python310:40:200
906448992211,9cyclictest1011705-21tail12:05:190
906448992211,9cyclictest1002552-21cpuspeed_turbos11:40:120
906448992210,11cyclictest908129-21awk07:15:000
139099220,4rtkit-daemon1389-21rtkit-daemon11:25:053
906458992119,1cyclictest3056-21PK-Backend12:21:103
906458992119,1cyclictest3056-21PK-Backend12:09:143
906458992119,1cyclictest3056-21PK-Backend12:02:593
906458992119,1cyclictest3056-21PK-Backend11:09:313
906458992119,1cyclictest3056-21PK-Backend07:22:533
906458992119,1cyclictest0-21swapper/308:33:243
906458992118,2cyclictest3056-21PK-Backend12:33:073
906458992118,2cyclictest3056-21PK-Backend10:58:173
906458992118,2cyclictest3056-21PK-Backend08:53:383
906458992118,2cyclictest3056-21PK-Backend08:42:233
906458992117,2cyclictest9417152sleep308:48:173
90645899210,20cyclictest3056-21PK-Backend12:36:463
90645899210,20cyclictest3056-21PK-Backend10:36:473
90645899210,19cyclictest3056-21PK-Backend09:55:103
90645899210,19cyclictest3056-21PK-Backend09:26:033
90645899210,19cyclictest3056-21PK-Backend08:21:203
906454992119,1cyclictest0-21swapper/212:27:062
906454992119,1cyclictest0-21swapper/211:24:072
906454992119,1cyclictest0-21swapper/210:45:132
906454992119,1cyclictest0-21swapper/210:20:532
906454992119,1cyclictest0-21swapper/209:44:032
906454992119,1cyclictest0-21swapper/208:26:152
906454992119,1cyclictest0-21swapper/208:16:302
906454992119,1cyclictest0-21swapper/207:53:592
906454992119,1cyclictest0-21swapper/207:42:432
906454992117,3cyclictest949671-21sed09:10:172
906454992113,6cyclictest956958-21awk09:30:172
90645499210,20cyclictest0-21swapper/212:21:262
90645499210,20cyclictest0-21swapper/211:48:472
90645499210,20cyclictest0-21swapper/208:46:382
90645499210,20cyclictest0-21swapper/208:11:022
90645499210,20cyclictest0-21swapper/208:03:032
90645499210,20cyclictest0-21swapper/207:22:362
90645499210,20cyclictest0-21swapper/207:10:462
90645499210,19cyclictest3056-21PK-Backend07:55:482
90645499210,19cyclictest0-21swapper/209:50:382
906451992115,4cyclictest976104-21find10:25:121
90644899219,11cyclictest0-21swapper/008:25:140
90644899219,10cyclictest944484-21missed_timers08:55:170
90644899215,14cyclictest3636880-21snapd08:10:350
906448992119,1cyclictest0-21swapper/010:17:520
906448992119,1cyclictest0-21swapper/009:13:230
906448992117,3cyclictest0-21swapper/007:51:160
906448992113,6cyclictest994613-21/usr/sbin/munin11:15:210
906448992112,7cyclictest957714-21cron09:35:010
906448992112,7cyclictest948128-21wc09:05:200
906448992111,9cyclictest980063-21taskset10:35:160
906448992111,8cyclictest939531-21python308:40:200
906448992111,8cyclictest932097-21/usr/sbin/munin08:20:150
906448992111,8cyclictest931458-21apt08:19:590
906448992111,8cyclictest924548-21awk08:00:000
906448992110,9cyclictest1013174-21diskstats12:10:150
90644899210,20cyclictest0-21swapper/010:31:170
90644899210,20cyclictest0-21swapper/008:04:360
90644899210,20cyclictest0-21swapper/007:22:570
906458992018,1cyclictest3056-21PK-Backend11:53:063
906458992018,1cyclictest3056-21PK-Backend11:37:433
906458992018,1cyclictest3056-21PK-Backend10:43:363
906458992018,1cyclictest3056-21PK-Backend07:52:493
906458992012,6cyclictest910608-21cron07:20:003
906458992011,6cyclictest968398-21sh10:05:003
90645899200,1cyclictest3056-21PK-Backend11:18:113
90645899200,1cyclictest3056-21PK-Backend10:21:333
90645899200,19cyclictest3056-21PK-Backend12:16:303
90645899200,19cyclictest3056-21PK-Backend11:59:203
90645899200,19cyclictest3056-21PK-Backend11:59:203
90645899200,19cyclictest3056-21PK-Backend11:12:093
90645899200,19cyclictest3056-21PK-Backend10:10:443
90645899200,19cyclictest3056-21PK-Backend07:58:023
90645899200,18cyclictest3056-21PK-Backend09:01:233
906454992018,1cyclictest0-21swapper/211:17:512
906454992018,1cyclictest0-21swapper/211:07:112
906454992018,1cyclictest0-21swapper/210:53:042
906454992018,1cyclictest0-21swapper/208:40:432
906454992018,1cyclictest0-21swapper/208:38:332
906454992018,1cyclictest0-21swapper/208:09:402
906454992018,1cyclictest0-21swapper/208:09:392
906454992018,1cyclictest0-21swapper/207:26:342
906454992018,1cyclictest0-21swapper/207:15:052
906454992015,4cyclictest982778-21cron10:45:012
90645499200,19cyclictest0-21swapper/211:11:362
90645499200,19cyclictest0-21swapper/210:10:402
90645499200,19cyclictest0-21swapper/208:32:172
906451992018,1cyclictest0-21swapper/107:48:391
906451992018,1cyclictest0-21swapper/107:30:551
906451992018,1cyclictest0-21swapper/107:10:331
906451992014,4cyclictest1095-21runrttasks08:40:211
906451992014,4cyclictest1016005-21sh12:20:001
906451992013,5cyclictest436-20systemd-journal10:59:541
906451992013,5cyclictest26-21ksoftirqd/109:20:201
906451992013,5cyclictest1001663-21sed11:35:211
906451992012,6cyclictest436-20systemd-journal09:20:001
90645199200,1cyclictest0-21swapper/111:14:021
90645199200,1cyclictest0-21swapper/110:11:361
90645199200,19cyclictest0-21swapper/108:26:181
90645199200,19cyclictest0-21swapper/108:22:121
90645199200,19cyclictest0-21swapper/107:25:211
90644899209,10cyclictest935360-21grep08:30:140
90644899208,10cyclictest996335-21cat11:20:230
906448992018,1cyclictest0-21swapper/010:06:160
906448992018,1cyclictest0-21swapper/010:06:150
906448992018,1cyclictest0-21swapper/007:30:080
906448992013,5cyclictest937106-21cpuspeed_turbos08:35:140
906448992013,5cyclictest436-20systemd-journal12:35:010
906448992012,7cyclictest916257-21tr07:35:130
906448992012,6cyclictest952267-21apt-get09:20:010
906448992012,6cyclictest1-21systemd08:50:000
906448992011,7cyclictest988063-21latency_hist11:00:010
906448992011,7cyclictest984521-21idleruntime-cro10:50:010
906448992011,6cyclictest945608-21cpu09:00:120
906448992010,8cyclictest965547-21df09:55:140
906448992010,8cyclictest951170-21/usr/sbin/munin09:15:130
90644899200,19cyclictest0-21swapper/010:00:280
906458991917,1cyclictest3056-21PK-Backend11:46:503
906458991917,1cyclictest3056-21PK-Backend11:00:543
906458991917,1cyclictest3056-21PK-Backend11:00:533
906458991917,1cyclictest3056-21PK-Backend08:01:413
906458991916,2cyclictest3056-21PK-Backend12:25:503
906458991916,2cyclictest3056-21PK-Backend11:30:403
90645899190,1cyclictest3056-21PK-Backend08:17:493
90645899190,1cyclictest3056-21PK-Backend08:11:483
90645899190,1cyclictest3056-21PK-Backend07:34:373
906454991917,1cyclictest0-21swapper/211:38:412
906454991917,1cyclictest0-21swapper/210:00:392
906454991917,1cyclictest0-21swapper/208:55:562
906454991917,1cyclictest0-21swapper/208:53:342
906454991917,1cyclictest0-21swapper/208:24:542
906454991913,4cyclictest988699-21if_enp2s011:00:192
906454991913,4cyclictest988699-21if_enp2s011:00:192
90645499190,1cyclictest0-21swapper/211:31:242
90645499190,1cyclictest0-21swapper/210:25:062
90645499190,1cyclictest0-21swapper/210:06:272
90645499190,1cyclictest0-21swapper/210:06:272
90645499190,1cyclictest0-21swapper/209:20:042
906451991917,1cyclictest0-21swapper/112:26:071
906451991917,1cyclictest0-21swapper/111:33:351
906451991917,1cyclictest0-21swapper/111:08:481
906451991917,1cyclictest0-21swapper/110:42:041
906451991917,1cyclictest0-21swapper/108:53:421
906451991917,1cyclictest0-21swapper/108:49:091
906451991917,1cyclictest0-21swapper/108:35:361
906451991917,1cyclictest0-21swapper/108:15:231
906451991914,4cyclictest26-21ksoftirqd/111:00:171
906451991914,4cyclictest26-21ksoftirqd/111:00:161
906451991912,5cyclictest26-21ksoftirqd/107:55:201
906451991912,5cyclictest1095-21runrttasks12:05:211
906451991911,6cyclictest965185-21apt09:55:131
906451991910,7cyclictest980448-21python310:35:201
90645199190,1cyclictest0-21swapper/112:21:281
90645199190,1cyclictest0-21swapper/111:27:191
90645199190,1cyclictest0-21swapper/109:31:071
90645199190,1cyclictest0-21swapper/108:00:561
90645199190,18cyclictest3056-21PK-Backend08:58:151
90645199190,18cyclictest0-21swapper/111:57:011
90645199190,18cyclictest0-21swapper/111:57:001
90645199190,18cyclictest0-21swapper/110:15:481
90645199190,18cyclictest0-21swapper/109:10:451
90645199190,18cyclictest0-21swapper/109:04:231
90645199190,18cyclictest0-21swapper/108:07:591
90645199190,18cyclictest0-21swapper/108:07:581
906448991912,5cyclictest1007566-21/usr/sbin/munin11:55:170
906448991912,5cyclictest1007566-21/usr/sbin/munin11:55:170
906448991910,7cyclictest960467-21/usr/sbin/munin09:40:160
906448991910,7cyclictest927806-21sshd08:07:510
906448991910,7cyclictest927806-21sshd08:07:510
906448991910,7cyclictest1000618-21/usr/sbin/munin11:35:120
90645899189,7cyclictest918330-21cat07:40:163
906458991816,1cyclictest3056-21PK-Backend07:10:313
906458991811,6cyclictest959732-21basename09:40:013
906454991816,1cyclictest0-21swapper/212:11:052
906454991816,1cyclictest0-21swapper/209:29:152
906454991811,5cyclictest1019935-21cut12:30:112
906451991816,1cyclictest0-21swapper/111:20:161
906451991816,1cyclictest0-21swapper/110:48:061
906451991816,1cyclictest0-21swapper/108:32:531
906451991816,1cyclictest0-21swapper/107:17:031
906451991813,3cyclictest1004101-21find11:45:101
906451991811,5cyclictest959705-21cron09:40:001
90645199180,17cyclictest0-21swapper/112:33:241
90645199180,10cyclictest9636812expr09:50:131
90645499179,6cyclictest913-21gdbus09:35:012
906454991715,1cyclictest0-21swapper/210:17:282
906454991711,4cyclictest2156-21gnome-shell07:35:092
906454991710,5cyclictest946115-21grep09:00:162
906454991710,5cyclictest1003448-21awk11:40:202
90645199179,6cyclictest966918-21apt10:00:111
90645199179,6cyclictest955383-21python309:25:201
906451991715,1cyclictest0-21swapper/111:52:341
906451991715,1cyclictest0-21swapper/111:15:031
906451991715,1cyclictest0-21swapper/108:12:381
906451991710,6cyclictest918241-21df07:40:131
90645899169,5cyclictest916943-21python307:35:213
906458991614,1cyclictest3056-21PK-Backend12:12:533
906458991614,1cyclictest3056-21PK-Backend07:49:243
906458991611,4cyclictest3056-21PK-Backend10:31:163
906454991614,1cyclictest0-21swapper/212:39:162
906454991612,3cyclictest948793-21latency_hist09:10:002
90645199169,5cyclictest962069-21expr09:45:141
90645199169,5cyclictest960412-21basename09:40:171
90645199168,6cyclictest916447-21df_inode07:35:131
90645199167,7cyclictest1022550-21users12:35:221
906451991614,1cyclictest0-21swapper/110:31:491
906451991610,4cyclictest985648-21seq10:50:211
90645899156,2cyclictest3056-21PK-Backend09:45:203
906458991513,1cyclictest3056-21PK-Backend10:52:153
906458991513,1cyclictest3056-21PK-Backend09:24:273
906458991513,1cyclictest3056-21PK-Backend08:35:343
906458991511,3cyclictest3056-21PK-Backend09:42:443
90645499154,10cyclictest962804-21awk09:45:232
906454991513,1cyclictest0-21swapper/212:00:572
906454991513,1cyclictest0-21swapper/210:57:312
90645199159,4cyclictest974306-21grep10:20:121
90645199159,4cyclictest969066-21idleruntime10:05:151
90645199159,4cyclictest969066-21idleruntime10:05:141
906458991410,3cyclictest3056-21PK-Backend10:05:173
906458991410,3cyclictest3056-21PK-Backend10:05:163
90645499149,4cyclictest34-21ksoftirqd/209:20:012
90645899139,3cyclictest3056-21PK-Backend09:15:413
90645899138,3cyclictest950530-21idleruntime-cro09:15:003
906458991311,1cyclictest3056-21PK-Backend07:25:173
90645899129,2cyclictest3056-21PK-Backend09:05:183
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional