You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-25 - 01:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sun May 24, 2026 12:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20386721230,3sleep20-21swapper/211:20:112
11504627740,14sleep00-21swapper/007:07:070
11492627740,13sleep30-21swapper/307:05:323
11520327246,16sleep10-21swapper/107:09:091
11512726344,13sleep20-21swapper/207:08:092
2161562570,3sleep30-21swapper/311:55:133
1814652520,2sleep13056-21PK-Backend10:15:131
2107152490,1sleep30-21swapper/311:40:013
1800132440,3sleep13056-21PK-Backend10:10:161
11542699320,19cyclictest147284-21taskset08:39:250
11542699310,16cyclictest0-21swapper/007:33:490
11542699310,16cyclictest0-21swapper/007:33:480
11543599294,13cyclictest0-21swapper/309:54:503
11543599293,17cyclictest215657-21taskset11:54:193
11543599293,16cyclictest175278-21taskset09:59:233
11543599293,10cyclictest185738-21taskset10:26:493
11543599292,18cyclictest189186-21taskset10:36:333
11543599292,16cyclictest125938-21taskset07:36:243
11543599284,8cyclictest222422-21taskset12:12:113
11543599283,9cyclictest138355-21taskset08:11:043
11543599283,16cyclictest194352-21taskset10:51:073
11543599283,10cyclictest190873-21taskset10:40:583
11543599283,10cyclictest170058-21taskset09:44:083
11543599282,13cyclictest143582-21taskset08:26:253
11543599282,13cyclictest129502-21taskset07:46:583
11543599282,12cyclictest200079-21taskset11:09:333
11542699280,15cyclictest0-21swapper/007:28:100
11542699280,15cyclictest0-21swapper/007:10:550
11542699280,12cyclictest0-21swapper/009:25:030
139099270,18rtkit-daemon3056-21PK-Backend11:28:041
11543599274,14cyclictest207036-21taskset11:29:543
11543599273,9cyclictest204987-21taskset11:20:473
11543599273,8cyclictest171549-21taskset09:46:003
11543599273,16cyclictest196560-21taskset10:58:533
11543599273,15cyclictest212500-21taskset11:45:093
11543599273,14cyclictest229237-21taskset12:30:323
11543599273,14cyclictest218867-21taskset12:01:023
11543599273,11cyclictest180597-21taskset10:12:353
11543599273,10cyclictest203581-21taskset11:19:583
11543599272,9cyclictest184010-21taskset10:21:533
11543599272,9cyclictest184010-21taskset10:21:523
11543599272,9cyclictest178892-21taskset10:07:573
11543599272,8cyclictest153916-21taskset08:55:363
11543599272,16cyclictest231248-21taskset12:39:063
11543599272,16cyclictest227600-21runrttasks12:26:563
11543599272,15cyclictest212007-21taskset11:41:563
11543599272,15cyclictest198190-21taskset11:02:353
11543599272,14cyclictest208660-21taskset11:33:273
11543599272,12cyclictest140116-21taskset08:16:253
115426992716,9cyclictest173854-21/usr/sbin/munin09:55:200
11542699271,24cyclictest206643-21timerwakeupswit11:25:230
11542699270,14cyclictest0-21swapper/011:30:220
2094942260,3chrt0-21swapper/211:35:152
11543599264,7cyclictest122420-21taskset07:25:453
11543599263,9cyclictest192612-21taskset10:46:023
11543599263,10cyclictest201642-21taskset11:12:223
11543599263,10cyclictest182469-21taskset10:19:203
11543599262,9cyclictest168347-21taskset09:39:223
11543599262,9cyclictest165170-21taskset09:30:033
11543599262,9cyclictest148859-21taskset08:42:243
11543599262,9cyclictest141951-21taskset08:22:443
11543599262,8cyclictest224309-21taskset12:19:093
11543599262,16cyclictest136702-21taskset08:07:083
11542699269,16cyclictest147886-21expr08:40:110
11542699260,14cyclictest0-21swapper/009:14:200
11542699260,13cyclictest0-21swapper/009:01:330
1580982250,3sleep00-21swapper/009:10:000
11543599252,9cyclictest127729-21taskset07:41:253
11543599252,14cyclictest225909-21taskset12:22:263
11543599252,14cyclictest187465-21runrttasks10:31:443
11543599252,14cyclictest150676-21taskset08:48:293
11543599252,11cyclictest152400-21taskset08:53:233
11543599252,10cyclictest220886-21runrttasks12:09:443
11542699258,16cyclictest166689-21turbostat09:34:590
115426992512,11cyclictest144545-21sh08:30:140
115426992511,12cyclictest190545-21cut10:40:190
11542699251,10cyclictest191199-21perf10:49:550
11542699248,15cyclictest126239-21file07:39:550
115426992413,10cyclictest171150-21ntp_states09:45:180
11542699240,23cyclictest0-21swapper/010:16:510
139099230,4rtkit-daemon3056-21PK-Backend08:51:181
139099230,15rtkit-daemon3056-21PK-Backend12:31:071
139099230,15rtkit-daemon0-21swapper/010:08:330
11543599233,8cyclictest177124-21taskset10:02:273
11543599232,8cyclictest164889-21runrttasks09:29:403
115428992320,2cyclictest3056-21PK-Backend11:57:321
11542699235,16cyclictest186608-21expr10:30:130
11542699232,17cyclictest175874-21apt10:00:100
139099220,15rtkit-daemon3056-21PK-Backend07:24:561
115432992220,1cyclictest0-21swapper/212:06:222
115432992220,1cyclictest0-21swapper/210:14:412
115432992214,6cyclictest436-20systemd-journal08:05:002
115432992214,6cyclictest201788-21sshd11:14:172
11543299220,20cyclictest192548-21users10:45:242
115428992220,1cyclictest3056-21PK-Backend11:43:531
115428992220,1cyclictest3056-21PK-Backend11:09:261
115428992220,1cyclictest3056-21PK-Backend09:40:301
115428992220,1cyclictest0-21swapper/108:37:101
115428992220,1cyclictest0-21swapper/107:36:141
11542899221,12cyclictest1862692apt10:30:111
11542899220,20cyclictest3056-21PK-Backend12:00:151
11542699228,12cyclictest141124-21awk08:20:160
11542699226,15cyclictest172555-21basename09:50:140
11542699226,14cyclictest231660-21ls12:40:010
11542699225,15cyclictest213179-21tr11:45:170
11542699222,17cyclictest161739-21apt09:20:120
11542699221,19cyclictest119940-21df_inode07:20:130
11542699220,20cyclictest0-21swapper/012:30:110
139099210,14rtkit-daemon3056-21PK-Backend11:18:281
139099210,14rtkit-daemon0-21swapper/109:26:501
139099210,14rtkit-daemon0-21swapper/107:27:101
139099210,14rtkit-daemon0-21swapper/011:58:410
139099210,14rtkit-daemon0-21swapper/011:50:110
11543599212,8cyclictest159714-21taskset09:14:563
115432992119,1cyclictest0-21swapper/212:30:082
115432992119,1cyclictest0-21swapper/211:07:212
115432992119,1cyclictest0-21swapper/210:33:192
115432992119,1cyclictest0-21swapper/210:18:062
115432992119,1cyclictest0-21swapper/209:49:132
115432992119,1cyclictest0-21swapper/209:40:342
115432992119,1cyclictest0-21swapper/209:30:412
115432992119,1cyclictest0-21swapper/208:40:592
115432992119,1cyclictest0-21swapper/208:26:182
115432992119,1cyclictest0-21swapper/208:12:402
115432992119,1cyclictest0-21swapper/207:38:262
115432992118,2cyclictest3056-21PK-Backend07:12:162
115432992117,3cyclictest1095-21runrttasks09:55:452
115432992114,5cyclictest168441-21systemctl09:40:002
115432992113,6cyclictest129271-21timerandwakeup07:45:212
11543299210,20cyclictest3056-21PK-Backend09:23:232
11543299210,20cyclictest0-21swapper/212:02:012
11543299210,20cyclictest0-21swapper/210:56:382
11543299210,20cyclictest0-21swapper/209:27:032
11543299210,20cyclictest0-21swapper/208:30:112
11543299210,20cyclictest0-21swapper/208:18:222
11543299210,19cyclictest883-21dbus-daemon07:25:002
11543299210,19cyclictest3056-21PK-Backend08:35:052
11543299210,19cyclictest3056-21PK-Backend07:56:102
11543299210,19cyclictest3056-21PK-Backend07:31:512
11543299210,19cyclictest3056-21PK-Backend07:31:502
115428992119,1cyclictest3056-21PK-Backend11:49:211
115428992119,1cyclictest3056-21PK-Backend11:38:121
115428992119,1cyclictest3056-21PK-Backend08:43:531
115428992119,1cyclictest0-21swapper/109:22:461
115428992119,1cyclictest0-21swapper/109:09:011
115428992119,1cyclictest0-21swapper/107:13:151
115428992118,2cyclictest3056-21PK-Backend08:46:441
11542899210,20cyclictest3056-21PK-Backend12:10:021
11542899210,20cyclictest3056-21PK-Backend11:11:351
11542899210,20cyclictest3056-21PK-Backend08:14:591
11542899210,19cyclictest3056-21PK-Backend10:20:331
11542899210,19cyclictest3056-21PK-Backend10:20:321
11542899210,19cyclictest3056-21PK-Backend09:48:071
11542899210,19cyclictest3056-21PK-Backend07:40:251
11542699218,12cyclictest2156-21gnome-shell07:49:080
11542699218,11cyclictest161532-21latency_hist09:20:010
11542699216,13cyclictest200956-21cat11:10:150
11542699215,14cyclictest202538-21basename11:15:150
115426992119,1cyclictest0-21swapper/012:13:550
115426992119,1cyclictest0-21swapper/012:02:200
115426992119,1cyclictest0-21swapper/009:37:210
115426992119,1cyclictest0-21swapper/008:00:550
11542699211,16cyclictest117834-21apt07:15:100
11542699210,20cyclictest168823-21tr09:40:130
11542699210,19cyclictest203988-21tr11:20:120
139099200,3rtkit-daemon3056-21PK-Backend10:36:071
139099200,14rtkit-daemon0-21swapper/109:14:191
139099200,13rtkit-daemon3056-21PK-Backend12:16:231
11543599206,12cyclictest134339-21wc08:00:183
11543599203,8cyclictest978-21snmpd07:12:263
115435992018,1cyclictest42-21ksoftirqd/307:34:123
115435992018,1cyclictest42-21ksoftirqd/307:34:113
115435992018,1cyclictest0-21swapper/307:16:143
11543599201,5cyclictest42-21ksoftirqd/308:37:473
11543599200,19cyclictest0-21swapper/309:18:083
11543599200,19cyclictest0-21swapper/309:00:173
11543599200,19cyclictest0-21swapper/307:59:343
11543599200,19cyclictest0-21swapper/307:52:303
115432992018,1cyclictest3056-21PK-Backend07:53:342
115432992018,1cyclictest0-21swapper/212:38:462
115432992018,1cyclictest0-21swapper/212:14:122
115432992018,1cyclictest0-21swapper/211:32:272
115432992018,1cyclictest0-21swapper/210:52:532
115432992018,1cyclictest0-21swapper/208:52:342
115432992018,1cyclictest0-21swapper/208:21:252
115432992018,1cyclictest0-21swapper/207:44:422
115432992013,6cyclictest150055-21ntpq08:45:172
115432992013,5cyclictest206834-21sshd11:27:152
11543299200,19cyclictest0-21swapper/212:22:502
11543299200,19cyclictest0-21swapper/211:46:482
11543299200,19cyclictest0-21swapper/210:38:412
11543299200,19cyclictest0-21swapper/210:22:192
11543299200,19cyclictest0-21swapper/210:22:192
11543299200,19cyclictest0-21swapper/208:58:162
115428992018,1cyclictest3056-21PK-Backend12:22:471
115428992018,1cyclictest3056-21PK-Backend11:32:311
115428992018,1cyclictest3056-21PK-Backend08:26:351
115428992018,1cyclictest3056-21PK-Backend08:22:441
115428992018,1cyclictest3056-21PK-Backend08:16:021
115428992018,1cyclictest0-21swapper/107:59:321
115428992017,2cyclictest3056-21PK-Backend10:52:161
115428992017,2cyclictest3056-21PK-Backend09:50:441
115428992013,6cyclictest145220-21/usr/sbin/munin08:30:201
11542899200,19cyclictest3056-21PK-Backend09:34:161
11542899200,19cyclictest3056-21PK-Backend07:49:041
11542899200,19cyclictest0-21swapper/109:15:301
11542899200,19cyclictest0-21swapper/108:02:081
11542899200,18cyclictest3056-21PK-Backend11:52:051
11542699207,12cyclictest0-21swapper/012:15:500
11542699206,12cyclictest0-21swapper/007:50:130
11542699205,13cyclictest132007-21basename07:55:120
115426992018,1cyclictest0-21swapper/012:21:060
115426992018,1cyclictest0-21swapper/011:02:580
115426992015,4cyclictest0-21swapper/008:08:050
11542699200,19cyclictest0-21swapper/010:56:090
11542699200,17cyclictest153183-21df_inode08:55:150
139099190,13rtkit-daemon0-21swapper/107:52:211
139099190,12rtkit-daemon3056-21PK-Backend10:04:171
115435991917,1cyclictest3056-21PK-Backend09:06:593
115435991917,1cyclictest0-21swapper/309:20:593
115432991917,1cyclictest3056-21PK-Backend09:06:482
115432991917,1cyclictest0-21swapper/210:42:262
115432991917,1cyclictest0-21swapper/210:04:412
115432991917,1cyclictest0-21swapper/209:51:022
115432991917,1cyclictest0-21swapper/208:06:392
115432991917,1cyclictest0-21swapper/207:18:122
115432991912,6cyclictest223884-21sed12:15:212
115432991911,7cyclictest184510-21sshd10:25:102
11543299190,1cyclictest0-21swapper/211:55:462
11543299190,1cyclictest0-21swapper/209:01:072
11543299190,18cyclictest0-21swapper/211:01:112
115428991917,1cyclictest3056-21PK-Backend12:28:281
115428991917,1cyclictest0-21swapper/109:38:421
115428991917,1cyclictest0-21swapper/109:01:431
115428991916,2cyclictest3056-21PK-Backend11:02:431
11542899190,1cyclictest3056-21PK-Backend12:06:311
11542899190,18cyclictest3056-21PK-Backend10:48:311
11542699197,11cyclictest2156-21gnome-shell08:27:090
11542699197,11cyclictest0-21swapper/010:35:180
11542699197,11cyclictest0-21swapper/008:53:290
11542699196,11cyclictest0-21swapper/008:15:120
11542699195,12cyclictest0-21swapper/010:20:110
11542699195,12cyclictest0-21swapper/010:20:110
11542699193,14cyclictest180799-21latency_hist10:15:000
11542699192,15cyclictest185405-21python310:25:200
115426991917,1cyclictest0-21swapper/012:08:070
115426991917,1cyclictest0-21swapper/011:37:380
11542699190,1cyclictest0-21swapper/008:14:120
11542699190,17cyclictest211156-21df11:40:120
11542699190,17cyclictest194158-21grep10:50:210
11542699190,17cyclictest149709-21df08:45:130
115435991816,1cyclictest3056-21PK-Backend07:24:123
115432991816,1cyclictest0-21swapper/211:18:092
115432991816,1cyclictest0-21swapper/210:09:402
115432991815,2cyclictest3056-21PK-Backend09:11:282
115428991816,1cyclictest3056-21PK-Backend08:58:531
115428991816,1cyclictest0-21swapper/107:18:351
11542699181,16cyclictest226404-21wc12:25:140
11542699180,16cyclictest198788-21grep11:05:140
11542699180,16cyclictest126697-21tr07:40:130
115435991715,1cyclictest3056-21PK-Backend08:31:113
115432991715,1cyclictest3056-21PK-Backend09:19:452
115432991715,1cyclictest0-21swapper/211:50:192
115432991714,2cyclictest0-21swapper/211:41:552
115432991710,5cyclictest227465-21awk12:25:232
115428991715,1cyclictest3056-21PK-Backend10:25:331
115428991715,1cyclictest3056-21PK-Backend09:58:211
115428991715,1cyclictest3056-21PK-Backend08:08:101
11543299169,5cyclictest121543-21expr07:25:132
115428991613,2cyclictest3056-21PK-Backend11:24:071
115428991513,1cyclictest3056-21PK-Backend10:56:021
115428991510,4cyclictest3056-21PK-Backend10:40:241
11542899149,4cyclictest3056-21PK-Backend10:08:531
115428991412,1cyclictest3056-21PK-Backend12:37:071
115428991410,3cyclictest123421-21forks07:30:151
115428991410,3cyclictest123421-21forks07:30:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional