You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-14 - 01:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sun Sep 13, 2026 12:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
48021221420,6sleep23056-21PK-Backend07:05:132
51414921280,4sleep13056-21PK-Backend08:40:011
50206221230,2sleep30-21swapper/308:05:143
48455921040,4sleep33056-21PK-Backend07:15:143
5556442820,3sleep2555646-21cat10:35:222
48093727344,20sleep10-21swapper/107:05:251
48147927241,24sleep00-21swapper/007:08:250
48132627040,24sleep30-21swapper/307:06:273
5633252380,2sleep13056-21PK-Backend10:59:541
5546982280,9sleep013-21ksoftirqd/010:35:120
5473312260,4sleep13056-21PK-Backend10:15:001
48173799261,23cyclictest514636-21hddtemp_smartct08:40:163
5767522250,4chrt3056-21PK-Backend11:35:223
5120462250,4sleep2331rcuc/208:35:002
481737992514,4cyclictest42-21ksoftirqd/310:15:293
481737992514,4cyclictest42-21ksoftirqd/310:15:293
4956072240,3sleep13056-21PK-Backend07:45:191
4904962240,6chrt0-21swapper/107:30:191
4904962240,6chrt0-21swapper/107:30:191
481737992416,6cyclictest42-21ksoftirqd/309:07:153
481734992414,8cyclictest522086-21python309:00:212
481737992318,3cyclictest42-21ksoftirqd/310:00:013
481737992317,5cyclictest978-21snmpd09:38:163
481737992313,3cyclictest42-21ksoftirqd/312:39:413
48173799230,3cyclictest538812-21/usr/sbin/munin09:50:183
481734992310,11cyclictest496353-21cron07:50:002
48173499230,22cyclictest0-21swapper/212:29:342
139099230,17rtkit-daemon3056-21PK-Backend12:29:343
48173799222,7cyclictest42-21ksoftirqd/308:55:153
48173799222,7cyclictest42-21ksoftirqd/307:45:003
481737992218,3cyclictest411rcuc/307:56:503
481737992214,6cyclictest1-21systemd09:30:003
48173799220,3cyclictest503935-21df08:10:153
481734992220,1cyclictest0-21swapper/210:00:512
481734992220,1cyclictest0-21swapper/209:22:042
481734992220,1cyclictest0-21swapper/207:30:512
481734992220,1cyclictest0-21swapper/207:30:512
48173499220,20cyclictest0-21swapper/212:17:322
48173499220,20cyclictest0-21swapper/207:40:042
481732992220,1cyclictest3056-21PK-Backend08:00:201
481727992220,1cyclictest0-21swapper/012:32:270
481727992220,1cyclictest0-21swapper/011:31:230
481727992220,1cyclictest0-21swapper/010:29:380
481727992220,1cyclictest0-21swapper/009:53:490
481727992220,1cyclictest0-21swapper/009:48:270
481727992220,1cyclictest0-21swapper/009:13:070
481727992220,1cyclictest0-21swapper/008:37:510
481727992220,1cyclictest0-21swapper/008:15:340
481727992220,1cyclictest0-21swapper/007:41:410
481727992215,5cyclictest587601-21cron12:10:010
139099220,14rtkit-daemon3056-21PK-Backend09:59:542
5712052210,4sleep342-21ksoftirqd/311:20:193
5712052210,4sleep342-21ksoftirqd/311:20:193
481737992119,1cyclictest0-21swapper/312:04:173
481737992119,1cyclictest0-21swapper/310:53:403
481737992119,1cyclictest0-21swapper/310:45:083
481737992119,1cyclictest0-21swapper/310:39:193
481737992118,2cyclictest411rcuc/307:51:573
481737992113,6cyclictest436-20systemd-journal12:00:003
481737992111,3cyclictest42-21ksoftirqd/311:09:303
481737992111,3cyclictest42-21ksoftirqd/307:10:033
48173799210,20cyclictest0-21swapper/310:57:383
48173799210,20cyclictest0-21swapper/309:21:183
481734992119,1cyclictest0-21swapper/212:08:062
481734992119,1cyclictest0-21swapper/211:57:522
481734992119,1cyclictest0-21swapper/211:47:512
481734992119,1cyclictest0-21swapper/211:41:152
481734992119,1cyclictest0-21swapper/211:17:162
481734992119,1cyclictest0-21swapper/210:56:072
481734992119,1cyclictest0-21swapper/210:50:322
481734992119,1cyclictest0-21swapper/210:45:452
481734992119,1cyclictest0-21swapper/210:26:462
481734992119,1cyclictest0-21swapper/208:26:482
481734992119,1cyclictest0-21swapper/207:28:482
481734992119,1cyclictest0-21swapper/207:10:092
481734992114,5cyclictest485644-21apt-get07:20:012
48173499210,20cyclictest0-21swapper/209:37:392
481732992119,1cyclictest3056-21PK-Backend08:24:131
481732992119,1cyclictest0-21swapper/111:27:321
481732992119,1cyclictest0-21swapper/111:19:141
481732992118,2cyclictest3056-21PK-Backend11:56:181
48173299210,20cyclictest3056-21PK-Backend08:10:141
48173299210,19cyclictest3056-21PK-Backend11:52:331
48173299210,19cyclictest3056-21PK-Backend09:40:181
481727992119,1cyclictest0-21swapper/012:02:460
481727992119,1cyclictest0-21swapper/011:44:210
481727992119,1cyclictest0-21swapper/008:56:580
481727992119,1cyclictest0-21swapper/008:21:150
48172799210,20cyclictest0-21swapper/012:18:010
48172799210,20cyclictest0-21swapper/010:51:220
48172799210,20cyclictest0-21swapper/009:41:460
48172799210,20cyclictest0-21swapper/007:10:450
5971292200,4sleep20-21swapper/212:35:122
5309712200,2sleep00-21swapper/009:25:230
48173799209,3cyclictest141rcu_preempt12:20:413
48173799209,3cyclictest141rcu_preempt12:20:403
48173799202,4cyclictest42-21ksoftirqd/309:45:183
481737992018,1cyclictest3056-21PK-Backend12:11:543
481737992018,1cyclictest0-21swapper/311:41:523
481737992018,1cyclictest0-21swapper/309:31:203
481737992018,1cyclictest0-21swapper/308:52:543
481737992018,1cyclictest0-21swapper/308:48:003
481737992016,3cyclictest411rcuc/309:02:143
481737992013,6cyclictest543034-21ntp_states10:00:183
481737992010,3cyclictest42-21ksoftirqd/312:32:213
48173799200,6cyclictest3056-21PK-Backend12:10:013
48173799200,1cyclictest0-21swapper/309:12:203
48173799200,19cyclictest3056-21PK-Backend11:25:433
48173799200,19cyclictest0-21swapper/311:01:373
48173799200,19cyclictest0-21swapper/310:30:213
48173799200,19cyclictest0-21swapper/310:22:103
48173799200,19cyclictest0-21swapper/310:22:103
48173799200,19cyclictest0-21swapper/309:44:583
48173799200,19cyclictest0-21swapper/307:20:543
48173799200,11cyclictest42-21ksoftirqd/308:37:093
48173499202,16cyclictest544697-21strings10:05:172
481734992018,1cyclictest0-21swapper/212:10:432
481734992018,1cyclictest0-21swapper/212:00:282
481734992018,1cyclictest0-21swapper/211:23:522
481734992018,1cyclictest0-21swapper/211:23:522
481734992018,1cyclictest0-21swapper/209:26:172
481734992018,1cyclictest0-21swapper/208:46:422
481734992018,1cyclictest0-21swapper/208:15:262
481734992018,1cyclictest0-21swapper/207:54:172
481734992018,1cyclictest0-21swapper/207:22:202
481734992015,3cyclictest505031-21turbostat.cron08:15:012
481734992012,6cyclictest514717-21awk08:40:172
48173499200,19cyclictest0-21swapper/211:00:062
48173499200,19cyclictest0-21swapper/209:51:182
48173499200,19cyclictest0-21swapper/208:00:052
481732992018,1cyclictest3056-21PK-Backend08:05:141
481732992018,1cyclictest3056-21PK-Backend07:24:581
481732992018,1cyclictest0-21swapper/112:30:111
48173299200,2cyclictest515108-21smartctl08:40:221
48173299200,19cyclictest3056-21PK-Backend10:22:431
48173299200,19cyclictest3056-21PK-Backend10:22:421
48173299200,19cyclictest3056-21PK-Backend09:11:521
48173299200,19cyclictest0-21swapper/112:27:001
48173299200,18cyclictest3056-21PK-Backend09:29:091
481727992018,1cyclictest0-21swapper/012:29:020
481727992018,1cyclictest0-21swapper/011:59:220
481727992018,1cyclictest0-21swapper/010:49:460
481727992018,1cyclictest0-21swapper/008:46:230
481727992018,1cyclictest0-21swapper/007:50:190
481727992018,1cyclictest0-21swapper/007:47:020
48172799200,19cyclictest3056-21PK-Backend09:55:250
48172799200,19cyclictest0-21swapper/009:04:210
48172799200,19cyclictest0-21swapper/008:54:540
48172799200,19cyclictest0-21swapper/008:42:310
48172799200,19cyclictest0-21swapper/008:25:550
48172799200,19cyclictest0-21swapper/008:01:280
48172799200,19cyclictest0-21swapper/007:56:550
48172799200,19cyclictest0-21swapper/007:25:320
48172799200,19cyclictest0-21swapper/007:22:410
48172799200,19cyclictest0-21swapper/007:18:560
139099200,3rtkit-daemon0-21swapper/210:30:142
139099200,14rtkit-daemon3056-21PK-Backend11:12:233
139099200,14rtkit-daemon0-21swapper/208:59:402
48173799196,3cyclictest141rcu_preempt11:32:383
48173799192,5cyclictest3056-21PK-Backend11:19:403
48173799192,4cyclictest3056-21PK-Backend12:19:483
481737991917,1cyclictest0-21swapper/308:22:393
481737991917,1cyclictest0-21swapper/307:26:083
481737991916,2cyclictest411rcuc/310:28:523
481737991915,3cyclictest508757-21find08:25:113
481737991915,3cyclictest411rcuc/310:41:233
481737991915,3cyclictest411rcuc/310:06:183
481737991915,3cyclictest411rcuc/309:17:143
481737991915,3cyclictest411rcuc/307:47:443
481737991915,3cyclictest411rcuc/307:31:003
481737991915,3cyclictest411rcuc/307:30:593
48173799191,4cyclictest500964-21python308:00:263
48173799190,5cyclictest2156-21gnome-shell07:35:093
48173799190,1cyclictest0-21swapper/308:15:563
481734991917,1cyclictest0-21swapper/209:42:122
481734991917,1cyclictest0-21swapper/207:55:322
481734991916,2cyclictest1095-21runrttasks07:37:322
481734991914,3cyclictest978-21snmpd08:50:022
481734991914,3cyclictest532157-21latency09:30:212
481734991912,5cyclictest537730-21cat09:45:202
481734991912,5cyclictest436-20systemd-journal10:45:002
48173499190,1cyclictest0-21swapper/212:20:092
48173499190,1cyclictest0-21swapper/212:20:092
48173499190,1cyclictest0-21swapper/208:36:082
48173499190,1cyclictest0-21swapper/208:06:532
481732991917,1cyclictest3056-21PK-Backend10:37:501
481732991917,1cyclictest0-21swapper/112:39:301
481732991917,1cyclictest0-21swapper/111:10:211
481732991917,1cyclictest0-21swapper/109:35:381
481732991917,1cyclictest0-21swapper/107:14:371
481732991916,2cyclictest3056-21PK-Backend07:17:551
481732991912,5cyclictest26-21ksoftirqd/111:40:151
48173299190,1cyclictest3056-21PK-Backend09:17:331
48173299190,18cyclictest0-21swapper/112:09:431
481727991917,1cyclictest0-21swapper/012:14:360
481727991917,1cyclictest0-21swapper/009:38:480
481727991917,1cyclictest0-21swapper/009:08:200
481727991917,1cyclictest0-21swapper/008:09:530
481727991916,2cyclictest3056-21PK-Backend07:30:250
481727991916,2cyclictest3056-21PK-Backend07:30:250
48172799190,1cyclictest0-21swapper/011:50:560
48172799190,1cyclictest0-21swapper/009:30:170
48172799190,17cyclictest568550-21munin-run11:15:000
5694032180,3sleep00-21swapper/011:15:180
481737991815,2cyclictest411rcuc/311:47:513
481737991815,2cyclictest411rcuc/308:30:303
48173799180,6cyclictest546123-21cat10:10:173
48173799180,16cyclictest581810-21python311:50:243
481734991816,1cyclictest0-21swapper/211:29:392
481734991816,1cyclictest0-21swapper/210:19:232
481734991816,1cyclictest0-21swapper/210:19:232
481734991811,6cyclictest576483-21ntp_states11:35:182
481734991811,5cyclictest524687-21cat09:10:112
481734991810,6cyclictest436-20systemd-journal09:05:272
481732991816,1cyclictest0-21swapper/112:24:371
481732991816,1cyclictest0-21swapper/112:24:371
481732991816,1cyclictest0-21swapper/110:18:231
481732991816,1cyclictest0-21swapper/110:18:231
481732991812,5cyclictest588908-21cat12:10:221
481732991812,4cyclictest2156-21gnome-shell12:17:081
48173299181,12cyclictest570551-21apt11:20:121
48173299181,12cyclictest570551-21apt11:20:111
48173299180,17cyclictest3056-21PK-Backend08:56:581
48172799189,7cyclictest572033-21awk11:25:000
48172799189,7cyclictest572033-21awk11:25:000
48172799189,7cyclictest0-21swapper/009:18:080
481727991816,1cyclictest0-21swapper/012:35:040
481727991816,1cyclictest0-21swapper/009:24:290
481727991816,1cyclictest0-21swapper/008:10:540
481727991815,2cyclictest562745-21smartctl10:55:200
481727991811,5cyclictest572671-21fw_packets11:25:140
139099180,12rtkit-daemon0-21swapper/211:31:202
48173499179,6cyclictest581691-21ntp_states11:50:232
481734991715,1cyclictest0-21swapper/212:31:242
481734991710,5cyclictest34-21ksoftirqd/211:05:112
481732991715,1cyclictest3056-21PK-Backend11:46:041
481732991715,1cyclictest3056-21PK-Backend11:00:211
481732991715,1cyclictest3056-21PK-Backend08:46:581
481732991715,1cyclictest3056-21PK-Backend08:16:011
481732991715,1cyclictest3056-21PK-Backend07:57:301
481732991715,1cyclictest0-21swapper/111:34:141
481732991713,2cyclictest511985-21latency_hist08:35:011
481732991710,6cyclictest576496-21ntpq11:35:201
48173299170,16cyclictest3056-21PK-Backend10:55:001
481727991715,1cyclictest0-21swapper/010:20:050
481727991715,1cyclictest0-21swapper/010:20:050
48172799171,12cyclictest591349-21apt12:20:120
48172799171,12cyclictest591349-21apt12:20:110
48172799170,16cyclictest547235-21basename10:15:000
48173499169,5cyclictest527071-21cat09:15:172
48173499168,6cyclictest545741-21sshd10:10:152
48173499167,7cyclictest0-21swapper/208:20:182
481734991612,3cyclictest567747-21grep11:10:212
481732991614,1cyclictest3056-21PK-Backend10:32:091
481732991614,1cyclictest3056-21PK-Backend09:49:371
481732991614,1cyclictest3056-21PK-Backend08:51:581
481732991613,2cyclictest3056-21PK-Backend10:25:401
481732991611,3cyclictest542153-21latency_hist10:00:011
48173299160,15cyclictest3056-21PK-Backend09:21:521
481727991614,1cyclictest0-21swapper/008:30:350
481727991610,4cyclictest553434-21awk10:30:150
48172799160,14cyclictest566819-21apt11:10:010
48173499159,5cyclictest550451-21ls10:20:222
48173499159,5cyclictest550451-21ls10:20:212
481732991513,1cyclictest3056-21PK-Backend09:53:431
481732991513,1cyclictest3056-21PK-Backend09:30:451
481732991511,3cyclictest3056-21PK-Backend09:01:101
48173299150,14cyclictest3056-21PK-Backend10:42:091
48173299150,14cyclictest3056-21PK-Backend08:29:471
48172799152,11cyclictest558029-21latency_hist10:45:010
48172799152,10cyclictest575749-21apt11:35:120
481727991510,3cyclictest565067-21cron11:05:000
481727991510,3cyclictest547929-21if_enp2s010:15:140
481727991510,3cyclictest547929-21if_enp2s010:15:130
48173299149,4cyclictest524519-21latency_hist09:10:011
481732991412,1cyclictest0-21swapper/111:09:341
48173299141,12cyclictest542966-21needreboot10:00:171
481732991411,2cyclictest3056-21PK-Backend07:28:231
48173299140,1cyclictest3056-21PK-Backend12:02:131
48173299140,1cyclictest3056-21PK-Backend10:49:471
48173299140,12cyclictest498099-21ls07:55:001
481727991412,1cyclictest3056-21PK-Backend10:00:110
481727991411,2cyclictest491341-21/usr/sbin/munin07:35:230
48173299139,3cyclictest3056-21PK-Backend07:36:211
481732991311,1cyclictest3056-21PK-Backend10:09:441
48173299130,5cyclictest3056-21PK-Backend07:40:191
48172799139,2cyclictest544174-21tr10:05:120
48172799136,6cyclictest1894998-21snapd11:49:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional