You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-31 - 15:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Tue Mar 31, 2026 12:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
308852721210,4sleep33056-21PK-Backend08:10:133
316004221180,3sleep00-21swapper/011:30:190
139099950,82rtkit-daemon0-21swapper/107:05:381
306518827947,22sleep20-21swapper/207:05:112
306517727739,9sleep30-21swapper/307:05:053
306517926243,13sleep00-21swapper/007:05:060
30843672560,4sleep234-21ksoftirqd/207:57:092
31092762500,2sleep33056-21PK-Backend09:10:003
31099112440,3sleep33056-21PK-Backend09:10:113
31442852410,8sleep2341ksoftirqd/210:45:222
30770052390,4sleep03056-21PK-Backend07:35:210
30868892380,4sleep03056-21PK-Backend08:05:130
31802332280,3sleep33056-21PK-Backend12:30:013
31731682270,2chrt0-21swapper/012:10:000
31731682270,2chrt0-21swapper/012:10:000
31286502260,13sleep234-21ksoftirqd/210:05:002
306679499257,16cyclictest3175330-21expr12:15:133
3066794992413,7cyclictest0-21swapper/312:00:123
306679299240,23cyclictest3079680-21apt07:45:112
306678799240,22cyclictest3140270-21ntpq10:35:171
306678499241,21cyclictest3153454-21apt-get11:14:590
31429322230,3sleep126-21ksoftirqd/110:45:011
3066792992314,7cyclictest1788610-21gdbus11:53:422
3066784992319,2cyclictest3102466-21apt08:50:090
3066784992317,4cyclictest3090936-21python308:15:210
3066784992317,4cyclictest3090936-21python308:15:200
139099230,3rtkit-daemon1389-21rtkit-daemon11:59:130
139099230,2rtkit-daemon1389-21rtkit-daemon10:19:230
139099230,2rtkit-daemon1389-21rtkit-daemon09:36:470
30980872220,3sleep20-21swapper/208:35:222
3066794992220,1cyclictest3056-21PK-Backend11:34:513
3066794992220,1cyclictest3056-21PK-Backend11:01:593
3066794992220,1cyclictest3056-21PK-Backend10:17:043
3066794992220,1cyclictest3056-21PK-Backend09:23:103
306679499220,21cyclictest0-21swapper/312:09:393
306679499220,20cyclictest3056-21PK-Backend10:42:383
306679499220,20cyclictest3056-21PK-Backend07:27:303
3066787992220,1cyclictest3056-21PK-Backend09:21:221
3066787992220,1cyclictest0-21swapper/110:50:311
3066787992220,1cyclictest0-21swapper/110:16:181
3066787992220,1cyclictest0-21swapper/109:18:251
3066787992220,1cyclictest0-21swapper/108:42:021
3066787992220,1cyclictest0-21swapper/107:45:161
306678799220,20cyclictest0-21swapper/112:00:401
3066784992215,5cyclictest3180021-21munin-run12:30:000
139099220,4rtkit-daemon1389-21rtkit-daemon12:01:010
139099220,4rtkit-daemon1389-21rtkit-daemon10:50:410
3066794992119,1cyclictest3056-21PK-Backend11:44:503
3066794992119,1cyclictest3056-21PK-Backend08:33:213
3066794992119,1cyclictest3056-21PK-Backend07:22:173
3066794992119,1cyclictest0-21swapper/312:13:433
3066794992119,1cyclictest0-21swapper/312:13:433
3066794992119,1cyclictest0-21swapper/307:43:063
3066794992119,1cyclictest0-21swapper/307:39:143
306679499211,8cyclictest3177667-21python312:20:213
3066794992118,2cyclictest3056-21PK-Backend09:33:583
3066794992118,2cyclictest3056-21PK-Backend08:15:023
3066794992118,2cyclictest3056-21PK-Backend08:15:023
3066794992110,6cyclictest42-21ksoftirqd/309:55:013
306679499210,20cyclictest3056-21PK-Backend10:14:063
306679499210,20cyclictest3056-21PK-Backend10:14:063
306679499210,20cyclictest0-21swapper/311:56:073
306679499210,1cyclictest3056-21PK-Backend07:13:043
306679499210,19cyclictest3056-21PK-Backend10:55:443
306679499210,19cyclictest3056-21PK-Backend10:47:523
306679499210,19cyclictest0-21swapper/310:22:113
3066792992119,1cyclictest0-21swapper/211:55:412
3066792992119,1cyclictest0-21swapper/209:15:082
306679299210,20cyclictest0-21swapper/209:57:532
306679299210,20cyclictest0-21swapper/209:22:242
306679299210,20cyclictest0-21swapper/208:52:222
3066787992119,1cyclictest3143972-21strings10:45:181
3066787992119,1cyclictest3056-21PK-Backend11:11:061
3066787992119,1cyclictest3056-21PK-Backend07:40:371
3066787992119,1cyclictest0-21swapper/111:46:081
3066787992119,1cyclictest0-21swapper/111:44:051
3066787992119,1cyclictest0-21swapper/111:23:231
3066787992119,1cyclictest0-21swapper/111:15:331
3066787992119,1cyclictest0-21swapper/110:13:201
3066787992119,1cyclictest0-21swapper/110:13:201
3066787992119,1cyclictest0-21swapper/109:01:341
3066787992119,1cyclictest0-21swapper/108:52:221
3066787992119,1cyclictest0-21swapper/108:39:381
3066787992119,1cyclictest0-21swapper/107:35:571
3066787992118,2cyclictest3056-21PK-Backend10:28:201
306678799210,20cyclictest0-21swapper/111:38:571
306678799210,20cyclictest0-21swapper/111:02:001
306678799210,20cyclictest0-21swapper/109:52:391
306678799210,20cyclictest0-21swapper/109:14:061
306678799210,20cyclictest0-21swapper/107:53:211
306678499210,20cyclictest3056-21PK-Backend10:01:070
306678499210,20cyclictest0-21swapper/010:49:200
306678499210,20cyclictest0-21swapper/010:37:150
306678499210,20cyclictest0-21swapper/009:03:070
306678499210,1cyclictest0-21swapper/010:23:580
139099210,15rtkit-daemon0-21swapper/007:32:210
30876542200,4sleep126-21ksoftirqd/108:05:211
306679499208,11cyclictest3120188-21/usr/sbin/munin09:40:153
3066794992018,1cyclictest3056-21PK-Backend10:39:153
3066794992018,1cyclictest3056-21PK-Backend09:36:083
3066794992018,1cyclictest3056-21PK-Backend08:42:263
3066794992018,1cyclictest0-21swapper/311:29:233
3066794992018,1cyclictest0-21swapper/309:18:023
3066794992017,2cyclictest3056-21PK-Backend12:37:093
3066794992017,2cyclictest3056-21PK-Backend11:46:533
3066794992017,2cyclictest3056-21PK-Backend11:37:553
3066794992017,2cyclictest3056-21PK-Backend11:18:553
3066794992017,2cyclictest0-21swapper/307:31:573
3066794992015,3cyclictest3136473-21diskstats10:25:133
3066794992013,6cyclictest3092290-21seq08:20:173
306679499200,7cyclictest3138557-21tr10:30:193
306679499200,1cyclictest3056-21PK-Backend08:36:193
306679499200,19cyclictest3056-21PK-Backend09:49:053
306679499200,19cyclictest0-21swapper/311:11:523
3066792992018,1cyclictest0-21swapper/209:51:102
3066792992018,1cyclictest0-21swapper/209:47:262
3066792992015,4cyclictest978-21snmpd10:40:092
3066792992013,5cyclictest3108099-21expr09:05:142
3066792992012,6cyclictest1-21systemd09:05:012
306679299200,3cyclictest3157197-21awk11:24:592
306679299200,1cyclictest0-21swapper/211:49:062
306679299200,1cyclictest0-21swapper/211:37:512
306679299200,1cyclictest0-21swapper/211:13:582
306679299200,1cyclictest0-21swapper/208:58:522
306679299200,1cyclictest0-21swapper/207:28:212
306679299200,19cyclictest0-21swapper/212:30:232
306679299200,19cyclictest0-21swapper/212:04:072
306679299200,19cyclictest0-21swapper/211:01:142
306679299200,19cyclictest0-21swapper/210:53:562
306679299200,19cyclictest0-21swapper/210:37:282
306679299200,19cyclictest0-21swapper/209:41:312
306679299200,19cyclictest0-21swapper/209:38:332
306679299200,19cyclictest0-21swapper/209:26:092
306679299200,19cyclictest0-21swapper/208:25:132
306679299200,19cyclictest0-21swapper/208:11:272
306679299200,19cyclictest0-21swapper/208:00:112
3066787992018,1cyclictest3056-21PK-Backend12:15:011
3066787992018,1cyclictest3056-21PK-Backend12:08:521
3066787992018,1cyclictest3056-21PK-Backend11:28:371
3066787992018,1cyclictest3056-21PK-Backend08:22:421
3066787992018,1cyclictest0-21swapper/112:25:001
3066787992018,1cyclictest0-21swapper/111:31:131
3066787992018,1cyclictest0-21swapper/110:55:441
3066787992018,1cyclictest0-21swapper/109:59:201
3066787992018,1cyclictest0-21swapper/108:57:291
3066787992017,2cyclictest3056-21PK-Backend07:33:071
3066787992013,5cyclictest3069525-21expr07:15:131
3066787992012,6cyclictest3174086-21ntpq12:10:211
3066787992012,6cyclictest3174086-21ntpq12:10:211
306678799200,1cyclictest0-21swapper/111:05:391
306678799200,19cyclictest3056-21PK-Backend10:08:131
306678799200,19cyclictest0-21swapper/112:21:571
306678799200,19cyclictest0-21swapper/110:33:411
306678799200,19cyclictest0-21swapper/108:45:131
306678799200,19cyclictest0-21swapper/108:28:231
306678799200,19cyclictest0-21swapper/108:16:261
306678799200,19cyclictest0-21swapper/108:16:261
306678799200,19cyclictest0-21swapper/107:28:411
306678499204,14cyclictest3150008-21awk11:05:000
3066784992018,1cyclictest0-21swapper/011:50:170
3066784992018,1cyclictest0-21swapper/011:48:140
3066784992018,1cyclictest0-21swapper/011:09:060
3066784992018,1cyclictest0-21swapper/009:07:590
3066784992018,1cyclictest0-21swapper/008:25:350
3066784992018,1cyclictest0-21swapper/007:51:270
3066784992018,1cyclictest0-21swapper/007:15:320
3066784992017,2cyclictest0-21swapper/009:10:090
3066784992016,3cyclictest3115592-21/usr/sbin/munin09:25:170
306678499200,1cyclictest3056-21PK-Backend12:21:120
306678499200,1cyclictest0-21swapper/009:30:240
306678499200,19cyclictest0-21swapper/012:38:560
306678499200,19cyclictest0-21swapper/012:30:520
306678499200,19cyclictest0-21swapper/011:26:510
306678499200,19cyclictest0-21swapper/010:42:150
306678499200,19cyclictest0-21swapper/010:34:530
306678499200,19cyclictest0-21swapper/009:17:130
306678499200,19cyclictest0-21swapper/008:57:390
306678499200,19cyclictest0-21swapper/008:11:020
139099200,16rtkit-daemon3066778-21cyclictest10:01:081
139099200,14rtkit-daemon0-21swapper/107:24:291
139099200,14rtkit-daemon0-21swapper/007:41:590
139099200,14rtkit-daemon0-21swapper/007:27:050
3066794991917,1cyclictest3056-21PK-Backend09:26:073
3066794991917,1cyclictest3056-21PK-Backend08:26:053
3066794991917,1cyclictest3056-21PK-Backend08:01:453
3066794991917,1cyclictest0-21swapper/311:05:373
3066794991916,2cyclictest3056-21PK-Backend07:53:133
3066794991916,2cyclictest3056-21PK-Backend07:18:523
3066794991911,6cyclictest3056-21PK-Backend08:49:593
306679499190,7cyclictest3128880-21/usr/sbin/munin10:05:133
306679499190,18cyclictest3056-21PK-Backend12:30:213
306679499190,18cyclictest3056-21PK-Backend11:21:333
306679499190,18cyclictest3056-21PK-Backend09:57:103
306679499190,18cyclictest3056-21PK-Backend07:55:023
3066792991917,1cyclictest3056-21PK-Backend07:50:382
3066792991917,1cyclictest0-21swapper/212:10:292
3066792991917,1cyclictest0-21swapper/212:10:292
3066792991917,1cyclictest0-21swapper/211:25:402
3066792991917,1cyclictest0-21swapper/210:15:502
3066792991917,1cyclictest0-21swapper/208:34:052
3066792991917,1cyclictest0-21swapper/208:23:022
3066792991917,1cyclictest0-21swapper/207:42:202
3066792991917,1cyclictest0-21swapper/207:38:272
3066792991917,1cyclictest0-21swapper/207:31:592
3066792991915,2cyclictest3100468-21idleruntime-cro08:45:002
3066792991914,3cyclictest978-21snmpd10:06:322
3066792991914,3cyclictest3069893-21needreboot07:15:182
3066792991913,5cyclictest3177328-21sed12:20:162
3066792991912,6cyclictest3172306-21sed12:05:232
3066792991911,6cyclictest883-21dbus-daemon08:50:012
306679299190,1cyclictest0-21swapper/210:12:062
306679299190,1cyclictest0-21swapper/210:12:062
306679299190,1cyclictest0-21swapper/208:15:462
306679299190,1cyclictest0-21swapper/208:15:462
306679299190,18cyclictest0-21swapper/211:17:362
306679299190,18cyclictest0-21swapper/210:21:452
3066787991917,1cyclictest0-21swapper/112:38:401
3066787991913,5cyclictest3116511-21wc09:30:011
3066787991911,6cyclictest3182017-21apt-get12:35:001
306678799190,1cyclictest3056-21PK-Backend10:21:241
306678799190,1cyclictest0-21swapper/109:38:051
306678799190,1cyclictest0-21swapper/107:59:021
306678799190,1cyclictest0-21swapper/107:10:491
3066784991917,1cyclictest0-21swapper/011:40:150
3066784991917,1cyclictest0-21swapper/010:26:210
3066784991917,1cyclictest0-21swapper/009:51:270
3066784991917,1cyclictest0-21swapper/008:30:420
3066784991917,1cyclictest0-21swapper/007:46:010
3066784991916,2cyclictest3148271-21munin-run11:00:010
3066784991913,4cyclictest13-21ksoftirqd/011:25:010
306678499191,14cyclictest3097874-21taskset08:35:190
3066784991910,7cyclictest883-21dbus-daemon09:45:000
306678499190,1cyclictest0-21swapper/009:21:320
306678499190,1cyclictest0-21swapper/008:21:160
139099190,14rtkit-daemon0-21swapper/007:24:220
3066794991816,1cyclictest3056-21PK-Backend08:54:243
3066794991813,3cyclictest3080072-21hddtemp_smartct07:45:153
3066792991816,1cyclictest0-21swapper/212:37:262
3066792991816,1cyclictest0-21swapper/212:29:072
3066792991816,1cyclictest0-21swapper/209:10:492
3066792991816,1cyclictest0-21swapper/207:21:182
3066792991815,2cyclictest3056-21PK-Backend11:30:072
3066787991816,1cyclictest0-21swapper/111:54:191
3066787991816,1cyclictest0-21swapper/109:41:031
3066787991814,3cyclictest3109298-21idleruntime-cro09:10:001
3066787991813,3cyclictest3122434-21kernelversion09:45:171
3066787991811,5cyclictest3095880-21cat08:30:171
306678799180,1cyclictest0-21swapper/109:30:131
3066784991813,3cyclictest3132160-21latency_hist10:15:010
3066784991813,3cyclictest3132160-21latency_hist10:15:000
3066784991813,3cyclictest3085520-21needreboot08:00:170
3066784991810,6cyclictest3125568-21grep09:55:130
3066794991715,1cyclictest3056-21PK-Backend08:58:293
3066794991712,3cyclictest3146533-21idleruntime-cro10:55:013
3066794991710,5cyclictest3088080-21turbostat08:10:003
3066792991715,1cyclictest0-21swapper/212:16:362
3066792991715,1cyclictest0-21swapper/209:34:142
3066792991711,4cyclictest3151893-21cut11:10:012
3066787991710,6cyclictest3089087-21ntp_states08:10:201
3066787991710,5cyclictest3085320-21cat08:00:151
306678499179,6cyclictest3122328-21cat09:45:150
306678499179,6cyclictest3099382-21entropy08:40:140
306678499179,6cyclictest3084751-21date08:00:010
306678499179,5cyclictest3161801-21python311:35:230
3066784991715,1cyclictest3056-21PK-Backend12:15:180
3066784991715,1cyclictest3056-21PK-Backend10:05:380
3066784991712,4cyclictest3067413-21ls07:10:150
306679499160,15cyclictest0-21swapper/310:02:413
3066792991610,4cyclictest3164118-21cron11:45:012
306678499169,5cyclictest3172976-21grep12:10:000
306678499161,10cyclictest3153454-21apt-get11:15:040
3066794991511,2cyclictest42-21ksoftirqd/309:00:153
306679299159,4cyclictest978-21snmpd07:13:342
306679299158,6cyclictest3136177-21cat10:25:122
306679299157,6cyclictest2173-21JS10:27:482
306678799159,4cyclictest1120-21snapd11:58:051
3066784991510,3cyclictest1095-21runrttasks08:45:450
306679499147,5cyclictest42-21ksoftirqd/311:50:153
306679299149,3cyclictest3086693-21cpuspeed08:05:122
306679299138,3cyclictest0-21swapper/210:30:162
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional