You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-09 - 22:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Thu Jul 09, 2026 12:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
98202022400,10sleep00-21swapper/007:05:140
101439521300,2sleep33056-21PK-Backend08:35:173
98310627236,14sleep10-21swapper/107:08:211
98303026445,13sleep30-21swapper/307:07:313
98289526444,14sleep20-21swapper/207:05:452
10584652490,1sleep00-21swapper/010:40:120
9940422470,2sleep00-21swapper/007:35:230
10635352350,2sleep00-21swapper/010:55:120
983392993010,6cyclictest1024676-21ls09:05:161
98339299299,6cyclictest1065335-21cpuspeed_turbos11:00:121
98339299299,4cyclictest1099592-21wc12:35:201
98339299298,7cyclictest0-21swapper/110:35:131
98339299298,6cyclictest1047976-21which10:10:161
98339299292,11cyclictest997991-21turbostat.cron07:50:001
983392992911,4cyclictest1080905-21tr11:45:131
98339299289,5cyclictest1062423-21ntpq10:50:191
98339299283,5cyclictest0-21swapper/108:40:111
98339299282,7cyclictest2156-21gnome-shell10:43:151
98339299282,5cyclictest0-21swapper/108:35:111
983392992813,13cyclictest992296-21timerwakeupswit07:30:231
98339299281,13cyclictest1036394-21apt-get09:40:031
10002812280,8sleep126-21ksoftirqd/107:55:121
983397992710,6cyclictest883-21dbus-daemon09:55:013
98339299278,5cyclictest1005409-21cat08:10:111
98339299277,4cyclictest1092209-21/usr/sbin/munin12:15:201
98339299277,4cyclictest0-21swapper/107:15:021
98339299272,6cyclictest2156-21gnome-shell08:03:141
98339299272,5cyclictest1075577-21apt11:30:121
983392992711,3cyclictest1033164-21acpi09:30:101
983392992711,13cyclictest1089428-21dump-pmu-power12:10:001
98339299269,3cyclictest1046036-21cstates10:05:121
98339299268,4cyclictest1077908-21sed11:35:161
98339299268,4cyclictest1055364-21sed10:30:161
98339299267,5cyclictest883-21dbus-daemon11:15:001
98339299267,5cyclictest1040262-21date09:50:011
98339299267,4cyclictest0-21swapper/108:25:141
98339299267,4cyclictest0-21swapper/108:25:141
98339299266,5cyclictest1093828-21which12:20:151
98339299263,7cyclictest983807-21apt-get07:10:061
98339299262,8cyclictest1034922-21apt09:35:091
98339299261,9cyclictest1067063-21cpuspeed_turbos11:05:121
983392992611,4cyclictest26-21ksoftirqd/111:50:141
98339299261,10cyclictest1044112-21apt10:00:121
98339799258,5cyclictest1095217-21cpuspeed12:25:123
98339799258,4cyclictest1006774-21ls08:15:013
98339799251,11cyclictest1026222-21apt-get09:10:043
98339299257,3cyclictest0-21swapper/112:00:131
98339299251,8cyclictest1095-21runrttasks08:05:221
98339299251,8cyclictest1085075-21/usr/sbin/munin11:55:151
98339299251,8cyclictest1085075-21/usr/sbin/munin11:55:151
98339299251,8cyclictest1022666-21grep09:00:121
98339299251,7cyclictest1053011-21/usr/sbin/munin10:25:131
983392992516,5cyclictest1042113-21idleruntime-cro09:55:011
98339299250,19cyclictest0-21swapper/112:33:041
139099250,3rtkit-daemon1389-21rtkit-daemon08:08:012
98339799240,1cyclictest0-21swapper/308:32:203
98339799240,19cyclictest3056-21PK-Backend12:33:043
98339799240,19cyclictest3056-21PK-Backend08:54:243
98339299247,4cyclictest1017293-21acpi08:45:101
98339299246,6cyclictest1007670-21ntpq08:15:201
98339299245,4cyclictest0-21swapper/109:20:001
98339299243,19cyclictest1079209-21cpuspeed_turbos11:40:121
98339299242,6cyclictest1060842-21python310:45:211
98339299242,6cyclictest1052165-21python310:20:221
98339299241,9cyclictest1090567-21python312:10:201
98339299241,8cyclictest996260-21awk07:45:001
98339299241,8cyclictest1072001-21latency_hist11:20:011
98339299241,7cyclictest1064927-21awk11:00:001
98339299241,7cyclictest1031837-21/usr/sbin/munin09:25:151
98339299241,7cyclictest1020949-21cpuspeed_turbos08:55:111
98339299240,8cyclictest1075504-21latency_hist11:30:011
98339299240,8cyclictest1072900-21wc11:20:181
98339299240,22cyclictest1012203-21cpuspeed08:30:121
983397992319,3cyclictest3056-21PK-Backend12:04:523
983397992319,3cyclictest3056-21PK-Backend11:39:583
983397992318,4cyclictest3056-21PK-Backend09:29:053
98339299231,7cyclictest988005-21/usr/sbin/munin07:20:131
98339299231,7cyclictest1008852-21runrttasks08:20:121
98339299231,5cyclictest1042351-21/usr/sbin/munin09:55:201
98339299230,7cyclictest1049466-21grep10:15:131
98339299230,21cyclictest1030095-21df_inode09:20:151
98339299230,21cyclictest1030095-21df_inode09:20:141
98338799233,6cyclictest984775-21/usr/sbin/munin07:10:200
10983972230,3chrt0-21swapper/012:35:000
983397992219,2cyclictest3056-21PK-Backend10:55:513
983397992217,4cyclictest0-21swapper/310:00:283
983397992210,3cyclictest3056-21PK-Backend12:07:413
98339799220,20cyclictest3056-21PK-Backend10:52:533
98339799220,20cyclictest3056-21PK-Backend07:31:093
98339799220,20cyclictest3056-21PK-Backend07:21:163
98339799220,20cyclictest3056-21PK-Backend07:16:503
983393992220,1cyclictest3056-21PK-Backend09:49:282
983393992220,1cyclictest0-21swapper/211:53:522
983393992220,1cyclictest0-21swapper/211:30:062
983393992220,1cyclictest0-21swapper/211:16:342
983393992220,1cyclictest0-21swapper/210:59:102
983393992220,1cyclictest0-21swapper/210:20:582
983393992220,1cyclictest0-21swapper/209:18:052
983393992220,1cyclictest0-21swapper/207:42:192
983393992220,1cyclictest0-21swapper/207:27:192
98339299222,8cyclictest2174-21JS10:27:481
98339299221,7cyclictest999713-21perf07:55:011
98339299221,7cyclictest992969-21/usr/sbin/munin07:35:201
98339299221,6cyclictest990387-21python307:25:211
98338799222,6cyclictest2156-21gnome-shell09:12:090
98338799221,6cyclictest1034340-21taskset09:31:100
983387992214,6cyclictest1092896-21apt-get12:20:000
98338799221,13cyclictest1082962-21seq11:50:130
98338799220,9cyclictest10503632cat10:15:230
139099220,4rtkit-daemon1389-21rtkit-daemon10:10:203
10431312220,3sleep23056-21PK-Backend09:55:192
983397992119,1cyclictest3056-21PK-Backend12:35:553
983397992119,1cyclictest3056-21PK-Backend11:55:193
983397992119,1cyclictest3056-21PK-Backend11:55:193
983397992119,1cyclictest3056-21PK-Backend11:03:553
983397992119,1cyclictest3056-21PK-Backend10:42:053
983397992119,1cyclictest3056-21PK-Backend09:36:353
983397992118,2cyclictest3056-21PK-Backend11:33:363
983397992118,2cyclictest3056-21PK-Backend07:58:133
98339799210,20cyclictest3056-21PK-Backend11:06:523
98339799210,20cyclictest3056-21PK-Backend10:36:243
98339799210,20cyclictest3056-21PK-Backend08:06:053
98339799210,20cyclictest3056-21PK-Backend07:11:163
98339799210,20cyclictest0-21swapper/308:16:183
98339799210,19cyclictest3056-21PK-Backend11:40:463
98339799210,19cyclictest3056-21PK-Backend11:24:503
98339799210,19cyclictest3056-21PK-Backend08:48:223
98339799210,19cyclictest3056-21PK-Backend08:21:323
98339799210,19cyclictest3056-21PK-Backend07:25:043
983393992119,1cyclictest0-21swapper/212:07:512
983393992119,1cyclictest0-21swapper/210:52:272
983393992119,1cyclictest0-21swapper/209:38:052
983393992119,1cyclictest0-21swapper/209:07:232
983393992119,1cyclictest0-21swapper/208:51:482
983393992119,1cyclictest0-21swapper/208:47:222
983393992119,1cyclictest0-21swapper/208:36:542
983393992119,1cyclictest0-21swapper/208:23:162
98339399210,20cyclictest0-21swapper/212:31:302
98339399210,20cyclictest0-21swapper/211:46:282
98339399210,20cyclictest0-21swapper/210:31:592
98339399210,20cyclictest0-21swapper/209:33:462
98339399210,20cyclictest0-21swapper/207:20:512
98338799215,9cyclictest10040931missed_timers08:05:170
98338799211,6cyclictest0-21swapper/009:40:000
983387992114,5cyclictest436-20systemd-journal07:43:240
983387992114,5cyclictest1009031-21sshd08:20:140
983387992113,6cyclictest998067-21cut07:50:000
983387992110,9cyclictest1057999-21unin-run10:40:000
98338799210,20cyclictest0-21swapper/009:58:290
983397992018,1cyclictest3056-21PK-Backend12:15:273
983397992018,1cyclictest3056-21PK-Backend10:06:093
983397992018,1cyclictest3056-21PK-Backend09:49:063
983397992018,1cyclictest3056-21PK-Backend09:18:233
983397992018,1cyclictest3056-21PK-Backend08:00:023
983397992018,1cyclictest3056-21PK-Backend07:41:173
983397992017,2cyclictest3056-21PK-Backend08:59:383
983397992017,2cyclictest3056-21PK-Backend07:45:563
983397992015,4cyclictest3056-21PK-Backend09:03:233
98339799200,1cyclictest3056-21PK-Backend10:25:023
98339799200,1cyclictest3056-21PK-Backend07:38:263
98339799200,19cyclictest3056-21PK-Backend12:20:063
98339799200,19cyclictest3056-21PK-Backend11:28:023
98339799200,19cyclictest0-21swapper/309:08:303
983393992018,1cyclictest0-21swapper/212:35:222
983393992018,1cyclictest0-21swapper/212:26:512
983393992018,1cyclictest0-21swapper/210:48:422
983393992018,1cyclictest0-21swapper/209:50:152
983393992018,1cyclictest0-21swapper/208:02:202
983393992018,1cyclictest0-21swapper/207:46:122
983393992014,5cyclictest1069219-21load11:10:182
983393992012,6cyclictest1059095-21python310:40:202
98339399200,1cyclictest0-21swapper/210:08:202
98339399200,19cyclictest0-21swapper/212:19:072
98339399200,19cyclictest0-21swapper/212:00:282
98339399200,19cyclictest0-21swapper/210:38:282
98339399200,19cyclictest0-21swapper/208:58:512
98339399200,19cyclictest0-21swapper/208:30:052
98339299200,1cyclictest0-21swapper/108:53:031
98338799209,10cyclictest1053700-21head10:25:180
98338799202,9cyclictest0-21swapper/010:10:370
983387992018,1cyclictest0-21swapper/012:36:400
983387992018,1cyclictest0-21swapper/011:12:030
983387992018,1cyclictest0-21swapper/011:00:140
983387992018,1cyclictest0-21swapper/010:51:220
983387992018,1cyclictest0-21swapper/009:51:260
983387992018,1cyclictest0-21swapper/009:24:290
983387992018,1cyclictest0-21swapper/009:24:290
983387992018,1cyclictest0-21swapper/009:19:080
983387992018,1cyclictest0-21swapper/008:04:130
983387992018,1cyclictest0-21swapper/007:58:580
983387992014,5cyclictest917-21in:imuxsock11:25:390
983387992013,5cyclictest1070954-21/usr/sbin/munin11:15:170
983387992012,7cyclictest1037516-21sed09:40:160
983387992012,7cyclictest1022798-21expr09:00:130
983387992012,6cyclictest436-20systemd-journal11:40:000
98338799200,1cyclictest0-21swapper/012:25:310
98338799200,19cyclictest161rcuc/012:07:470
98338799200,19cyclictest0-21swapper/011:41:510
98338799200,19cyclictest0-21swapper/011:08:180
98338799200,19cyclictest0-21swapper/008:55:220
98338799200,19cyclictest0-21swapper/008:47:310
98338799200,19cyclictest0-21swapper/008:37:170
98338799200,19cyclictest0-21swapper/008:27:170
98338799200,19cyclictest0-21swapper/008:27:170
98338799200,19cyclictest0-21swapper/007:25:250
10304332200,6sleep3411rcuc/309:20:193
10304332200,6sleep3411rcuc/309:20:193
983397991917,1cyclictest3056-21PK-Backend11:17:543
983397991917,1cyclictest3056-21PK-Backend10:31:323
983397991917,1cyclictest3056-21PK-Backend09:58:183
983397991914,1cyclictest3056-21PK-Backend10:48:343
98339799190,1cyclictest3056-21PK-Backend11:14:573
98339799190,1cyclictest3056-21PK-Backend10:19:073
98339799190,1cyclictest3056-21PK-Backend08:40:453
983393991917,1cyclictest3056-21PK-Backend11:03:432
983393991917,1cyclictest0-21swapper/210:29:502
983393991917,1cyclictest0-21swapper/210:15:032
983393991917,1cyclictest0-21swapper/208:15:262
983393991917,1cyclictest0-21swapper/207:57:542
983393991917,1cyclictest0-21swapper/207:31:592
983393991917,1cyclictest0-21swapper/207:12:112
983393991912,6cyclictest1073571-21cron11:25:002
983393991910,7cyclictest1074202-21df11:25:132
98339399190,1cyclictest0-21swapper/212:22:592
98339399190,1cyclictest0-21swapper/211:41:282
98339399190,1cyclictest0-21swapper/208:42:562
98339399190,1cyclictest0-21swapper/207:15:222
98339399190,18cyclictest0-21swapper/209:28:592
98339299193,3cyclictest0-21swapper/109:10:211
983387991917,1cyclictest3056-21PK-Backend08:13:380
983387991917,1cyclictest0-21swapper/010:22:290
983387991917,1cyclictest0-21swapper/007:52:560
983387991913,4cyclictest2156-21gnome-shell12:21:090
983387991913,4cyclictest1032128-21cut09:25:180
983387991912,6cyclictest1082243-21apt-get11:50:000
983387991911,6cyclictest1040111-21apt-get09:50:000
98338799190,1cyclictest0-21swapper/010:04:110
98338799190,1cyclictest0-21swapper/008:50:080
98338799190,1cyclictest0-21swapper/007:21:470
98338799190,18cyclictest0-21swapper/010:33:170
98338799190,18cyclictest0-21swapper/008:40:420
983397991816,1cyclictest3056-21PK-Backend09:30:403
983397991815,2cyclictest3056-21PK-Backend10:20:423
983397991815,2cyclictest3056-21PK-Backend07:50:363
983393991816,1cyclictest0-21swapper/210:14:152
983393991816,1cyclictest0-21swapper/209:43:472
983393991816,1cyclictest0-21swapper/208:25:052
983393991816,1cyclictest0-21swapper/208:25:052
983393991816,1cyclictest0-21swapper/207:53:282
983393991811,5cyclictest1030168-21awk09:20:162
983393991811,5cyclictest1030168-21awk09:20:162
983387991816,1cyclictest0-21swapper/007:32:420
983387991814,3cyclictest1087407-21python312:01:250
983387991813,3cyclictest1007434-21latency08:15:170
983387991811,6cyclictest1012214-21ls08:30:120
983387991810,6cyclictest1072522-21ls11:20:140
10680782180,4sleep234-21ksoftirqd/211:06:242
983397991715,1cyclictest3056-21PK-Backend09:44:123
98339399171,7cyclictest141rcu_preempt12:15:012
983393991715,1cyclictest0-21swapper/207:39:292
983393991711,5cyclictest1027309-21sed09:10:222
98338799178,7cyclictest986550-21/usr/sbin/munin07:15:160
983387991715,1cyclictest0-21swapper/011:30:420
983387991711,5cyclictest1046390-21ls10:05:160
983387991711,5cyclictest1025289-21sed09:05:220
983387991710,6cyclictest1084754-21cat11:55:100
983387991710,6cyclictest1084754-21cat11:55:100
983387991710,5cyclictest1089850-21grep12:10:110
10844902170,3chrt0-21swapper/211:55:092
10844902170,3chrt0-21swapper/211:55:092
983393991614,1cyclictest0-21swapper/209:00:402
983393991614,1cyclictest0-21swapper/208:12:482
983387991614,1cyclictest0-21swapper/010:48:240
983397991410,3cyclictest42-21ksoftirqd/311:45:223
983397991410,3cyclictest3056-21PK-Backend12:14:023
983397991410,3cyclictest1011481-21chrt08:25:263
983397991410,3cyclictest1011481-21chrt08:25:263
98339399148,4cyclictest1078220-21grep11:35:202
983397991310,2cyclictest3056-21PK-Backend11:51:203
983393991311,1cyclictest0-21swapper/210:01:512
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional