You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-25 - 18:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Mon May 25, 2026 12:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
68145522450,14sleep2331rcuc/207:05:102
139099770,67rtkit-daemon0-21swapper/307:05:563
68142927347,16sleep00-21swapper/007:05:080
68286527237,13sleep10-21swapper/107:09:311
7299822500,4sleep33056-21PK-Backend09:20:133
6985472470,3sleep33056-21PK-Backend07:50:203
7356862400,3sleep20-21swapper/209:35:162
68304299374,18cyclictest0-21swapper/207:59:452
68304299374,18cyclictest0-21swapper/207:59:452
68304299373,19cyclictest0-21swapper/211:58:082
68304299373,19cyclictest0-21swapper/211:58:072
68304299291,4cyclictest883-21dbus-daemon11:35:012
683042992912,15cyclictest746519-21awk10:05:182
683039992917,8cyclictest436-20systemd-journal12:25:001
7304922280,3sleep00-21swapper/009:20:180
683042992813,13cyclictest788029-21expr12:05:122
683042992813,13cyclictest771001-21grep11:15:172
683042992812,14cyclictest714951-21cat08:35:212
68304299278,17cyclictest684715-21/usr/sbin/munin07:15:012
7514012260,8sleep234-21ksoftirqd/210:20:152
683046992618,5cyclictest729644-21idleruntime-cro09:20:013
68304299269,15cyclictest777543-21expr11:35:122
68304299268,16cyclictest883-21dbus-daemon11:00:002
68304299268,16cyclictest883-21dbus-daemon11:00:002
68304299263,21cyclictest699574-21sed07:55:002
139099260,4rtkit-daemon1389-21rtkit-daemon09:17:592
7233942250,4sleep20-21swapper/209:00:152
68304299258,15cyclictest737992-21sshd09:40:292
68304299256,17cyclictest774316-21latency11:25:152
68304299256,17cyclictest685567-21cat07:15:112
68303699250,24cyclictest696901-21/usr/sbin/munin07:45:180
68304699240,22cyclictest686536-21awk07:15:213
68304299247,15cyclictest749084-21cut10:15:012
68304299246,16cyclictest758583-21df_inode10:40:142
68304299246,16cyclictest753154-21tr10:25:122
683042992410,12cyclictest34-21ksoftirqd/208:00:212
68304299240,4cyclictest2085813-21snapd09:59:132
68303699245,5cyclictest436-20systemd-journal11:40:000
683036992416,6cyclictest796983-21df12:30:120
7459012230,4chrt3056-21PK-Backend10:05:133
683046992321,1cyclictest3056-21PK-Backend08:50:193
683046992315,6cyclictest796092-21cron12:30:003
68304699231,4cyclictest42-21ksoftirqd/312:34:543
68304299239,12cyclictest717451-21cut08:45:002
68304299237,15cyclictest688172-21sed07:20:212
68304299236,15cyclictest706361-21cat08:10:212
683042992321,1cyclictest0-21swapper/209:21:442
683042992312,9cyclictest767485-21missed_timers11:05:192
683042992310,11cyclictest978-21snmpd12:17:242
683039992320,2cyclictest3056-21PK-Backend10:56:521
683039992320,2cyclictest3056-21PK-Backend10:56:521
68303999231,20cyclictest747977-21basename10:10:151
68303699231,20cyclictest787423-21munin-run12:04:590
68303699230,21cyclictest792222-21users12:15:230
683046992220,1cyclictest0-21swapper/309:57:583
683046992220,1cyclictest0-21swapper/307:58:283
683046992220,1cyclictest0-21swapper/307:58:283
683046992219,2cyclictest3056-21PK-Backend12:17:443
683046992219,2cyclictest3056-21PK-Backend07:47:333
68304699220,20cyclictest3056-21PK-Backend12:14:193
68304699220,20cyclictest0-21swapper/310:10:163
68304299228,12cyclictest2156-21gnome-shell07:43:082
68304299227,14cyclictest0-21swapper/208:05:112
68304299226,14cyclictest762222-21grep10:50:162
68304299224,16cyclictest798622-21cat12:35:132
68304299223,17cyclictest766161-21timerwakeupswit11:00:232
683042992220,1cyclictest0-21swapper/210:17:002
683042992220,1cyclictest0-21swapper/208:25:412
683042992220,1cyclictest0-21swapper/207:32:072
68304299221,8cyclictest744560-21/usr/sbin/munin10:00:162
683042992217,4cyclictest0-21swapper/212:31:252
68304299221,2cyclictest0-21swapper/211:20:112
683039992220,1cyclictest3056-21PK-Backend08:04:231
683039992220,1cyclictest0-21swapper/110:20:091
683036992217,3cyclictest690008-21users07:25:240
683036992213,8cyclictest780610-21cut11:45:010
683046992119,1cyclictest3056-21PK-Backend07:35:293
683046992119,1cyclictest0-21swapper/311:03:553
683046992119,1cyclictest0-21swapper/310:35:583
683046992119,1cyclictest0-21swapper/310:30:373
683046992119,1cyclictest0-21swapper/309:49:263
683046992119,1cyclictest0-21swapper/308:06:113
683046992114,6cyclictest883-21dbus-daemon12:10:003
68304699210,20cyclictest3056-21PK-Backend11:48:373
68304699210,20cyclictest3056-21PK-Backend07:31:453
68304699210,20cyclictest0-21swapper/310:28:143
68304699210,20cyclictest0-21swapper/308:01:383
68304299216,14cyclictest1095-21runrttasks09:47:502
68304299216,13cyclictest754721-21/usr/sbin/munin10:30:112
68304299214,5cyclictest734174-21cut09:30:182
68304299213,3cyclictest0-21swapper/212:11:302
68304299213,3cyclictest0-21swapper/212:04:132
68304299213,3cyclictest0-21swapper/208:57:312
68304299213,3cyclictest0-21swapper/208:57:312
68304299213,3cyclictest0-21swapper/207:39:402
68304299212,16cyclictest726440-21apt09:10:112
683042992119,1cyclictest0-21swapper/212:28:002
683042992119,1cyclictest0-21swapper/211:42:302
683042992119,1cyclictest0-21swapper/210:36:132
68304299211,19cyclictest712527-21cstates08:30:132
68304299210,20cyclictest0-21swapper/211:48:252
68304299210,20cyclictest0-21swapper/207:45:582
68304299210,19cyclictest741826-21latency_hist09:55:012
68304299210,19cyclictest3056-21PK-Backend08:46:012
683039992119,1cyclictest3056-21PK-Backend12:06:491
683039992119,1cyclictest3056-21PK-Backend12:00:531
683039992119,1cyclictest0-21swapper/112:32:231
683039992119,1cyclictest0-21swapper/112:27:231
683039992119,1cyclictest0-21swapper/111:54:111
683039992119,1cyclictest0-21swapper/111:46:071
683039992119,1cyclictest0-21swapper/111:19:311
683039992119,1cyclictest0-21swapper/110:18:341
683039992119,1cyclictest0-21swapper/109:43:041
683039992119,1cyclictest0-21swapper/109:06:141
683039992119,1cyclictest0-21swapper/108:40:191
683039992119,1cyclictest0-21swapper/108:24:571
683039992119,1cyclictest0-21swapper/108:18:341
683039992118,2cyclictest3056-21PK-Backend07:56:381
683039992118,2cyclictest3056-21PK-Backend07:56:381
683039992114,4cyclictest766606-21idleruntime-cro11:04:591
68303999210,1cyclictest0-21swapper/107:53:271
68303999210,19cyclictest3056-21PK-Backend08:11:051
68303999210,19cyclictest0-21swapper/111:20:181
683036992113,6cyclictest1-21systemd11:50:000
68303699210,20cyclictest0-21swapper/008:33:370
68303699210,19cyclictest3056-21PK-Backend10:58:170
68303699210,19cyclictest3056-21PK-Backend10:58:170
68303699210,19cyclictest0-21swapper/010:15:030
68303699210,19cyclictest0-21swapper/008:37:370
139099210,4rtkit-daemon1389-21rtkit-daemon10:46:592
683046992018,1cyclictest3056-21PK-Backend12:24:203
683046992018,1cyclictest3056-21PK-Backend10:16:573
683046992018,1cyclictest3056-21PK-Backend07:42:593
683046992018,1cyclictest0-21swapper/312:00:263
683046992018,1cyclictest0-21swapper/311:24:243
683046992018,1cyclictest0-21swapper/310:58:493
683046992018,1cyclictest0-21swapper/310:58:493
683046992018,1cyclictest0-21swapper/309:51:023
683046992018,1cyclictest0-21swapper/308:35:453
683046992013,6cyclictest776527-21sed11:30:203
68304699200,1cyclictest3056-21PK-Backend08:58:163
68304699200,1cyclictest3056-21PK-Backend08:58:163
68304699200,19cyclictest3056-21PK-Backend11:56:423
68304699200,19cyclictest3056-21PK-Backend11:56:423
68304699200,19cyclictest3056-21PK-Backend08:17:273
68304699200,19cyclictest3056-21PK-Backend07:12:033
68304699200,19cyclictest0-21swapper/312:39:543
68304699200,19cyclictest0-21swapper/310:41:183
68304699200,18cyclictest3056-21PK-Backend11:16:193
68304299206,12cyclictest793297-21idleruntime12:20:162
68304299203,2cyclictest0-21swapper/207:28:212
683042992018,1cyclictest0-21swapper/211:12:432
683042992018,1cyclictest0-21swapper/208:16:542
68304299200,6cyclictest733124-21awk09:30:002
68304299200,5cyclictest783592-21chrt11:50:372
68303999202,12cyclictest761679-21apt10:50:121
683039992018,1cyclictest3056-21PK-Backend11:26:121
683039992018,1cyclictest3056-21PK-Backend08:05:101
683039992018,1cyclictest0-21swapper/112:17:231
683039992018,1cyclictest0-21swapper/112:12:231
683039992018,1cyclictest0-21swapper/110:30:021
683039992018,1cyclictest0-21swapper/107:39:351
683039992018,1cyclictest0-21swapper/107:32:041
683039992013,5cyclictest689438-21cut07:25:171
68303999200,1cyclictest0-21swapper/111:37:151
68303999200,19cyclictest3056-21PK-Backend09:58:461
68303999200,19cyclictest0-21swapper/110:45:511
683036992018,1cyclictest0-21swapper/012:28:470
683036992018,1cyclictest0-21swapper/009:26:020
683036992018,1cyclictest0-21swapper/007:51:270
683036992013,6cyclictest722691-21cron08:59:590
683036992013,6cyclictest722691-21cron08:59:590
683036992012,6cyclictest784576-21df11:55:130
683036992012,6cyclictest784576-21df11:55:120
68303699200,19cyclictest0-21swapper/012:24:350
68303699200,19cyclictest0-21swapper/012:11:100
68303699200,19cyclictest0-21swapper/011:28:110
68303699200,19cyclictest0-21swapper/010:53:100
68303699200,19cyclictest0-21swapper/010:11:180
68303699200,19cyclictest0-21swapper/009:48:200
68303699200,19cyclictest0-21swapper/008:24:520
68303699200,19cyclictest0-21swapper/008:17:090
68303699200,19cyclictest0-21swapper/007:57:350
68303699200,19cyclictest0-21swapper/007:57:340
68303699200,19cyclictest0-21swapper/007:40:450
68303699200,19cyclictest0-21swapper/007:39:570
68303699200,19cyclictest0-21swapper/007:30:170
68303699200,19cyclictest0-21swapper/007:19:570
68303699200,19cyclictest0-21swapper/007:10:500
683046991917,1cyclictest0-21swapper/310:01:103
68304699190,1cyclictest3056-21PK-Backend11:51:353
68304699190,1cyclictest3056-21PK-Backend10:47:463
68304699190,1cyclictest0-21swapper/308:12:553
68304699190,18cyclictest3056-21PK-Backend07:21:583
68304299195,12cyclictest720051-21sed08:50:182
68304299195,12cyclictest0-21swapper/208:22:432
68304299191,4cyclictest725748-21grep09:05:222
68303999192,15cyclictest780490-21perf11:45:011
683039991917,1cyclictest3056-21PK-Backend10:36:101
683039991917,1cyclictest0-21swapper/109:51:501
683039991913,4cyclictest1095-21runrttasks09:00:141
68303999190,1cyclictest3056-21PK-Backend11:34:171
68303999190,1cyclictest3056-21PK-Backend07:13:331
68303999190,1cyclictest0-21swapper/110:42:531
68303999190,1cyclictest0-21swapper/110:06:291
68303999190,1cyclictest0-21swapper/107:44:081
68303999190,1cyclictest0-21swapper/107:21:301
68303999190,1cyclictest0-21swapper/107:16:511
683036991917,1cyclictest0-21swapper/010:45:520
683036991917,1cyclictest0-21swapper/010:36:130
683036991917,1cyclictest0-21swapper/010:07:340
683036991917,1cyclictest0-21swapper/009:32:250
683036991913,5cyclictest729667-21systemctl09:20:000
683036991913,4cyclictest720451-21threads08:50:220
683036991912,5cyclictest735820-21ntpq09:35:180
683036991911,6cyclictest768878-21cat11:10:120
68303699190,1cyclictest3056-21PK-Backend10:00:380
68303699190,1cyclictest3056-21PK-Backend09:55:170
68303699190,1cyclictest3056-21PK-Backend08:40:480
68303699190,1cyclictest3056-21PK-Backend08:29:380
68303699190,1cyclictest3056-21PK-Backend08:10:260
68303699190,18cyclictest0-21swapper/011:50:150
68303699190,18cyclictest0-21swapper/010:34:380
68303699190,18cyclictest0-21swapper/009:53:400
7391112180,3sleep10-21swapper/109:45:181
7350142180,2sleep33056-21PK-Backend09:35:023
683046991816,1cyclictest3056-21PK-Backend11:42:083
683046991816,1cyclictest0-21swapper/311:28:103
683046991816,1cyclictest0-21swapper/309:41:303
683046991816,1cyclictest0-21swapper/308:34:573
683046991811,5cyclictest716417-21cat08:40:173
68304699180,6cyclictest778236-21cat11:35:203
68303999189,6cyclictest715511-21grep08:40:011
683039991816,1cyclictest3056-21PK-Backend10:01:091
683039991816,1cyclictest0-21swapper/110:25:301
683039991816,1cyclictest0-21swapper/109:35:061
683039991814,3cyclictest0-21swapper/111:12:141
683036991816,1cyclictest0-21swapper/010:23:210
683036991816,1cyclictest0-21swapper/009:08:190
683036991816,1cyclictest0-21swapper/007:22:400
683036991811,6cyclictest727105-21grep09:10:180
683036991811,5cyclictest718874-21sshd08:45:300
683036991811,5cyclictest704212-21load08:05:170
68303699180,1cyclictest0-21swapper/009:44:210
683046991715,1cyclictest3056-21PK-Backend07:26:043
683046991714,2cyclictest3056-21PK-Backend11:12:003
683046991711,4cyclictest752479-21sh10:25:003
683039991715,1cyclictest3056-21PK-Backend11:57:561
683039991715,1cyclictest3056-21PK-Backend11:57:561
683039991715,1cyclictest0-21swapper/107:47:191
683039991712,4cyclictest26-21ksoftirqd/108:45:101
683039991712,3cyclictest767671-21grep11:05:221
683039991711,5cyclictest726811-21df09:10:131
683039991710,5cyclictest733248-21cut09:30:011
683036991715,1cyclictest0-21swapper/011:02:350
683036991715,1cyclictest0-21swapper/010:42:550
683036991712,3cyclictest703118-21latency_hist08:05:000
68303999169,5cyclictest713296-21grep08:30:211
68303999168,6cyclictest710646-21cat08:25:141
68303999163,8cyclictest733373-21apt09:30:101
68303999162,9cyclictest728174-21apt09:15:121
683039991614,1cyclictest0-21swapper/108:50:521
68303999161,11cyclictest729911-21apt09:20:141
683039991610,4cyclictest721911-21strings08:55:181
683039991610,4cyclictest721911-21strings08:55:181
68303699169,6cyclictest767157-21expr11:05:130
683036991614,1cyclictest0-21swapper/012:07:580
683036991614,1cyclictest0-21swapper/011:23:520
683036991613,2cyclictest3056-21PK-Backend11:31:080
683036991611,4cyclictest1095-21runrttasks11:15:130
68303699161,11cyclictest722976-21apt09:00:120
68304699158,6cyclictest883-21dbus-daemon08:30:003
68304699157,6cyclictest42-21ksoftirqd/309:10:123
683046991513,1cyclictest3056-21PK-Backend08:47:083
68303699150,13cyclictest798493-21cpuspeed_turbos12:35:120
68304699149,4cyclictest3056-21PK-Backend09:28:193
683046991411,2cyclictest3056-21PK-Backend11:05:073
683046991410,3cyclictest3056-21PK-Backend10:50:513
683046991410,3cyclictest3056-21PK-Backend09:37:303
68304699140,12cyclictest3056-21PK-Backend08:20:123
683039991412,1cyclictest3056-21PK-Backend12:39:481
683036991412,1cyclictest0-21swapper/010:27:550
68304699139,3cyclictest3056-21PK-Backend09:00:123
683046991311,1cyclictest3056-21PK-Backend09:05:403
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional