You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-22 - 13:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackcslot7s.osadl.org (updated Tue Sep 22, 2026 00:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
108662922260,6sleep00-21swapper/019:05:170
112764721160,4sleep30-21swapper/321:00:193
114361221120,4sleep13056-21PK-Backend21:45:161
108749427938,13sleep20-21swapper/219:08:432
108611327342,22sleep30-21swapper/319:05:063
10859172690,8chrt0-21swapper/119:05:021
11680122510,2sleep20-21swapper/222:55:142
10923572490,2sleep10-21swapper/119:20:121
11976852470,2sleep00-21swapper/000:20:010
11785862430,4sleep13056-21PK-Backend23:25:161
10942252400,3sleep20-21swapper/219:25:132
10995642330,5sleep10-21swapper/119:40:161
139099260,4rtkit-daemon1389-21rtkit-daemon00:18:163
11371152260,10sleep234-21ksoftirqd/221:25:192
139099250,4rtkit-daemon1389-21rtkit-daemon22:14:582
139099250,4rtkit-daemon1389-21rtkit-daemon22:14:572
139099240,3rtkit-daemon1389-21rtkit-daemon23:42:322
139099240,3rtkit-daemon1389-21rtkit-daemon19:54:493
1087747992219,2cyclictest3056-21PK-Backend22:16:473
1087747992219,2cyclictest3056-21PK-Backend21:56:133
1087747992217,3cyclictest1124007-21hddtemp_smartct20:50:153
108774799221,19cyclictest1163277-21ntp_states22:40:193
108774799220,20cyclictest3056-21PK-Backend19:20:593
1087742992220,1cyclictest0-21swapper/223:23:122
108773799220,7cyclictest1150139-21munin-run22:04:591
1087735992220,1cyclictest0-21swapper/022:49:330
1087735992220,1cyclictest0-21swapper/021:32:210
1087735992220,1cyclictest0-21swapper/021:21:050
1087735992219,2cyclictest3056-21PK-Backend19:52:450
1087735992219,2cyclictest0-21swapper/023:39:080
108773599220,21cyclictest0-21swapper/022:40:070
1087747992119,1cyclictest3056-21PK-Backend20:38:523
1087747992119,1cyclictest0-21swapper/322:23:303
1087747992119,1cyclictest0-21swapper/322:11:543
1087747992119,1cyclictest0-21swapper/322:11:543
1087747992119,1cyclictest0-21swapper/322:07:003
1087747992119,1cyclictest0-21swapper/321:36:043
1087747992119,1cyclictest0-21swapper/321:20:233
1087747992119,1cyclictest0-21swapper/320:30:563
1087747992119,1cyclictest0-21swapper/319:56:153
1087747992119,1cyclictest0-21swapper/300:39:573
1087747992118,2cyclictest3056-21PK-Backend00:03:343
108774799211,7cyclictest1169469-21apt23:00:093
1087747992115,4cyclictest2156-21gnome-shell23:59:093
1087747992114,6cyclictest1115654-21ntpq20:25:193
1087747992113,6cyclictest1136659-21cat21:25:143
108774799210,2cyclictest1088414-21irqstats19:10:173
108774799210,20cyclictest0-21swapper/321:34:163
108774799210,20cyclictest0-21swapper/319:19:233
108774799210,19cyclictest3056-21PK-Backend19:45:263
108774799210,19cyclictest3056-21PK-Backend00:12:333
108774799210,19cyclictest1200339-21python300:25:223
108774799210,19cyclictest0-21swapper/323:20:083
1087742992119,1cyclictest3056-21PK-Backend23:30:162
1087742992114,5cyclictest1189115-21expr23:55:122
108774299210,20cyclictest0-21swapper/221:06:592
1087735992119,1cyclictest0-21swapper/023:43:140
1087735992119,1cyclictest0-21swapper/023:07:250
1087735992119,1cyclictest0-21swapper/022:11:210
1087735992119,1cyclictest0-21swapper/022:11:210
1087735992119,1cyclictest0-21swapper/021:40:110
1087735992119,1cyclictest0-21swapper/021:05:440
1087735992119,1cyclictest0-21swapper/020:17:380
1087735992119,1cyclictest0-21swapper/019:35:330
1087735992119,1cyclictest0-21swapper/019:29:120
108773599211,18cyclictest1089413-21turbostat19:14:590
108773599210,20cyclictest0-21swapper/023:54:440
108773599210,20cyclictest0-21swapper/021:46:130
108773599210,20cyclictest0-21swapper/021:38:220
108773599210,20cyclictest0-21swapper/020:37:110
108773599210,20cyclictest0-21swapper/020:04:400
108773599210,20cyclictest0-21swapper/019:30:480
108773599210,20cyclictest0-21swapper/000:16:130
139099200,17rtkit-daemon0-21swapper/322:03:153
108774799202,4cyclictest1202335-21python300:30:233
1087747992018,1cyclictest3056-21PK-Backend23:29:213
1087747992018,1cyclictest3056-21PK-Backend00:05:373
1087747992018,1cyclictest3056-21PK-Backend00:05:373
1087747992018,1cyclictest0-21swapper/323:51:043
1087747992018,1cyclictest0-21swapper/323:36:303
1087747992018,1cyclictest0-21swapper/321:41:183
1087747992018,1cyclictest0-21swapper/320:00:143
1087747992012,6cyclictest1158194-21python322:25:213
108774799200,19cyclictest0-21swapper/323:45:433
108774799200,19cyclictest0-21swapper/323:30:213
108774799200,19cyclictest0-21swapper/322:35:333
108774799200,19cyclictest0-21swapper/321:46:323
108774799200,19cyclictest0-21swapper/321:10:363
108774299203,15cyclictest1091882-21dump-pmu-power19:19:592
1087742992018,1cyclictest0-21swapper/223:16:562
1087742992013,6cyclictest1187251-21expr23:50:142
1087742992013,5cyclictest436-20systemd-journal00:35:002
1087742992013,5cyclictest1146338-21sshd21:54:212
1087742992012,6cyclictest1200975-21unin-run00:30:012
1087742992012,6cyclictest1197215-21apt-get00:20:002
1087742992012,6cyclictest1169642-21expr23:00:122
1087742992012,6cyclictest1119127-21ntp_kernel_pll_20:35:192
1087742992012,6cyclictest1108456-21cat20:05:182
1087737992018,1cyclictest0-21swapper/119:14:111
1087737992012,6cyclictest883-21dbus-daemon19:20:011
108773799200,19cyclictest3056-21PK-Backend22:55:211
108773799200,19cyclictest3056-21PK-Backend22:30:001
108773799200,19cyclictest0-21swapper/119:25:061
1087735992018,1cyclictest3056-21PK-Backend00:30:320
1087735992018,1cyclictest0-21swapper/020:59:090
1087735992018,1cyclictest0-21swapper/020:30:490
1087735992018,1cyclictest0-21swapper/019:46:560
1087735992018,1cyclictest0-21swapper/000:26:260
1087735992013,5cyclictest1175916-21sshd23:17:230
1087735992012,4cyclictest1089893-21apt-key19:15:010
108773599200,1cyclictest0-21swapper/022:36:490
108773599200,1cyclictest0-21swapper/000:12:070
108773599200,19cyclictest0-21swapper/021:52:150
108773599200,19cyclictest0-21swapper/021:27:080
108773599200,19cyclictest0-21swapper/019:40:200
108773599200,19cyclictest0-21swapper/019:22:160
108773599200,19cyclictest0-21swapper/000:38:300
11716852190,4sleep13056-21PK-Backend23:05:191
11169362190,10sleep234-21ksoftirqd/220:30:152
108774799191,7cyclictest0-21swapper/320:05:233
1087747991917,1cyclictest3056-21PK-Backend23:16:023
1087747991917,1cyclictest0-21swapper/322:54:543
1087747991917,1cyclictest0-21swapper/322:49:323
1087747991917,1cyclictest0-21swapper/300:20:453
1087747991914,4cyclictest978-21snmpd22:55:543
1087747991914,3cyclictest1171574-21df_inode23:05:163
108774799191,16cyclictest1134007-21perf21:20:003
1087747991911,6cyclictest1110393-21tr20:10:173
1087747991911,6cyclictest1098821-21cron19:40:003
108774799190,1cyclictest3056-21PK-Backend19:26:203
108774799190,1cyclictest0-21swapper/321:54:233
108774299197,10cyclictest1166747-21ntp_states22:50:182
1087742991917,1cyclictest0-21swapper/222:44:592
1087742991917,1cyclictest0-21swapper/222:27:492
1087742991917,1cyclictest0-21swapper/221:19:082
1087742991917,1cyclictest0-21swapper/200:09:082
1087742991917,1cyclictest0-21swapper/200:09:082
1087742991913,4cyclictest1179591-21cron23:30:002
1087742991911,3cyclictest34-21ksoftirqd/221:20:012
108773799193,14cyclictest1159751-21grep22:30:221
1087737991917,1cyclictest3056-21PK-Backend20:30:361
1087737991917,1cyclictest0-21swapper/121:55:531
1087735991912,5cyclictest1159462-21cat22:30:190
108773599191,16cyclictest1177875-21perf23:25:010
1087735991910,7cyclictest1-21systemd00:00:000
108773599190,1cyclictest0-21swapper/022:04:250
108773599190,1cyclictest0-21swapper/000:04:570
108773599190,18cyclictest0-21swapper/022:18:030
108773599190,18cyclictest0-21swapper/021:58:310
139099180,15rtkit-daemon953-21gmain19:51:572
1087747991812,4cyclictest1113638-21cat20:20:163
1087747991812,4cyclictest1095-21runrttasks20:15:123
1087747991810,6cyclictest1129932-21python321:05:203
1087742991817,1cyclictest0-21swapper/222:49:252
1087742991816,1cyclictest0-21swapper/222:06:482
1087742991816,1cyclictest0-21swapper/222:04:582
1087742991816,1cyclictest0-21swapper/220:13:452
1087742991816,1cyclictest0-21swapper/219:48:502
1087742991816,1cyclictest0-21swapper/219:43:442
1087742991816,1cyclictest0-21swapper/219:35:542
1087742991816,1cyclictest0-21swapper/219:22:342
1087742991815,2cyclictest1171205-21apt23:05:142
1087742991815,2cyclictest0-21swapper/220:45:212
1087742991813,3cyclictest1198191-21if_enp2s000:20:162
1087742991813,3cyclictest1139727-21acpi21:35:112
1087742991812,5cyclictest1203586-21entropy00:35:152
1087742991812,4cyclictest1147135-21wc21:55:132
1087742991812,4cyclictest1114670-21cron20:29:552
1087742991811,5cyclictest1185775-21awk23:45:162
1087742991811,5cyclictest1142124-21grep21:40:182
1087742991810,7cyclictest1127827-21ntp_states21:00:212
108773799182,14cyclictest1103617-21/usr/sbin/munin19:50:211
1087737991816,1cyclictest0-21swapper/122:50:081
1087737991816,1cyclictest0-21swapper/121:15:591
1087737991811,5cyclictest1180134-21apt23:30:121
1087737991811,5cyclictest1155515-21cut22:20:001
108773799180,16cyclictest1186795-21cron23:50:001
1087735991816,1cyclictest0-21swapper/023:25:020
1087735991816,1cyclictest0-21swapper/022:28:380
1087735991813,3cyclictest1193750-21sh00:10:000
1087735991813,3cyclictest1193750-21sh00:10:000
1087735991811,5cyclictest1155942-21ls22:20:120
108774799179,6cyclictest1183947-21cat23:40:113
1087747991714,2cyclictest0-21swapper/323:15:003
1087747991711,5cyclictest1120130-21cat20:40:113
108774799170,15cyclictest1122619-21ntp_states20:45:193
1087742991715,1cyclictest0-21swapper/222:24:242
1087742991715,1cyclictest0-21swapper/221:12:402
1087742991715,1cyclictest0-21swapper/220:56:572
1087742991712,4cyclictest2175-21JS10:27:482
1087742991712,3cyclictest1114684-21turbostat.cron20:25:002
1087742991711,5cyclictest1088240-21expr19:10:142
1087742991710,6cyclictest1153868-21apt22:15:122
1087737991712,4cyclictest1134705-21cpuspeed_turbos21:20:131
1087737991712,3cyclictest1095-21runrttasks22:20:171
108773799171,14cyclictest1130551-21grep21:09:591
1087737991710,5cyclictest436-20systemd-journal19:39:201
1087735991715,1cyclictest0-21swapper/022:07:290
1087735991715,1cyclictest0-21swapper/020:45:230
1087735991712,4cyclictest1113855-21needreboot20:20:190
1087735991711,5cyclictest1104867-21wc19:55:160
1087747991614,1cyclictest3056-21PK-Backend19:44:383
1087747991612,3cyclictest3056-21PK-Backend19:31:293
1087747991611,3cyclictest1159053-21/usr/sbin/munin22:30:173
108774299169,5cyclictest436-20systemd-journal23:14:552
108774299169,5cyclictest1123735-21sort20:50:122
108774299167,3cyclictest1162056-21sshd22:37:262
1087742991614,1cyclictest3056-21PK-Backend19:30:112
1087742991614,1cyclictest0-21swapper/200:10:572
1087742991612,2cyclictest34-21ksoftirqd/223:35:112
1087742991611,3cyclictest34-21ksoftirqd/200:00:202
1087742991611,3cyclictest1139476-21perf21:35:002
1087742991610,4cyclictest1111658-21grep20:15:132
108773799169,5cyclictest1191436-21wc00:00:231
108773799169,5cyclictest1151108-21ntpq22:05:181
108773799169,5cyclictest1101374-21cat19:45:171
108773799168,6cyclictest1170465-21cat23:00:201
1087737991614,1cyclictest0-21swapper/123:18:271
1087737991613,2cyclictest3056-21PK-Backend21:27:201
1087737991612,3cyclictest1202135-21grep00:30:201
1087737991612,3cyclictest1192255-21/usr/sbin/munin00:05:131
1087737991612,3cyclictest1192255-21/usr/sbin/munin00:05:121
1087737991611,3cyclictest1190259-21sh23:59:591
108773799161,11cyclictest1194018-21apt00:10:141
1087737991610,5cyclictest1161005-21ls22:35:121
1087737991610,5cyclictest1126920-21sed21:00:011
108773599168,6cyclictest1108152-21grep20:05:130
108773599162,10cyclictest1179613-21apt-get23:30:030
1087735991612,3cyclictest1110173-21if_enp2s020:10:150
108773599161,13cyclictest1185512-21cpuspeed_turbos23:45:120
1087735991611,3cyclictest1132714-21tr21:15:120
1087735991611,3cyclictest1120234-21cpuspeed20:40:120
1087735991610,4cyclictest1168352-21grep22:55:180
11085932150,4sleep13056-21PK-Backend20:05:181
108774799159,4cyclictest1126023-21cut20:55:183
108774299158,6cyclictest1143669-21cat21:45:162
108774299157,7cyclictest1159215-21tr22:30:172
1087742991511,2cyclictest34-21ksoftirqd/219:55:122
108773799159,5cyclictest1165735-21cron22:50:001
108773799159,5cyclictest1122433-21cat20:45:181
108773799159,4cyclictest1152227-21cpuspeed_turbos22:10:141
108773799159,4cyclictest1152227-21cpuspeed_turbos22:10:131
108773799159,4cyclictest1120235-21cpuspeed20:40:121
108773799159,4cyclictest1095630-21cpu19:30:131
108773799158,6cyclictest0-21swapper/120:35:231
108773799157,6cyclictest1105749-21cron20:00:001
108773799151,2cyclictest141rcu_preempt21:15:001
1087737991510,4cyclictest1125159-21ls20:55:021
1087737991510,3cyclictest1187582-21latency23:50:191
1087737991510,3cyclictest1111530-21cpuspeed20:15:131
108773799150,1cyclictest3056-21PK-Backend00:28:501
108773799150,14cyclictest3056-21PK-Backend23:37:051
1087735991510,4cyclictest1123757-21tr20:50:120
1087735991510,4cyclictest1115962-21grep20:25:230
1087742991411,2cyclictest0-21swapper/220:43:132
108773799149,3cyclictest1106083-21cpuspeed20:00:151
108773799143,7cyclictest1127083-21apt21:00:131
1087737991412,1cyclictest3056-21PK-Backend00:23:021
1087737991412,1cyclictest0-21swapper/123:40:311
1087737991410,3cyclictest3056-21PK-Backend00:35:351
1087737991410,3cyclictest3056-21PK-Backend00:17:131
108773799140,1cyclictest3056-21PK-Backend20:28:261
108773799140,13cyclictest3056-21PK-Backend23:11:381
108773799140,13cyclictest3056-21PK-Backend21:52:561
108773799140,13cyclictest3056-21PK-Backend21:34:371
108773599149,4cyclictest3056-21PK-Backend22:50:110
108773599149,4cyclictest1173435-21sed23:10:160
108773599149,4cyclictest1127240-21grep21:00:140
1087735991411,2cyclictest1169443-21/usr/sbin/munin23:00:160
108773799139,3cyclictest3056-21PK-Backend20:23:551
108773799131,2cyclictest141rcu_preempt21:35:111
1087737991311,1cyclictest3056-21PK-Backend23:21:511
1087737991310,2cyclictest3056-21PK-Backend20:14:351
108773799130,1cyclictest3056-21PK-Backend21:40:321
108773799128,3cyclictest3056-21PK-Backend22:41:581
108773599128,3cyclictest1095-21runrttasks21:10:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional