You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-01 - 17:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot5s.osadl.org (updated Tue Jul 01, 2025 12:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1207513995641,11cyclictest121ktimers/009:45:130
1207528995239,9cyclictest391ktimers/307:45:103
1207523995018,10cyclictest1241580-21sed08:45:262
1207528994942,5cyclictest1291270-21kworker/u8:2+events_unbound11:10:183
1207528994940,5cyclictest1291425-21sh11:09:593
1207513994942,4cyclictest1226844-21awk08:05:010
1207528994838,6cyclictest199-21systemd-network12:40:003
1207523994732,11cyclictest311ktimers/210:45:152
1207515994733,11cyclictest231ktimers/108:30:171
1207528994639,4cyclictest1245814-21apt-key09:00:013
1207513994616,27cyclictest121ktimers/007:15:150
1207513994616,27cyclictest121ktimers/007:15:140
1207515994537,5cyclictest1-21systemd11:45:251
1207515994536,5cyclictest1277364-21munin-plugin-st10:30:011
1207523994436,5cyclictest1279070-21grep10:35:002
1207523994417,9cyclictest1295565-21cat11:20:192
1207523994417,10cyclictest1220864-21taskset07:45:212
1207523994416,10cyclictest1233795-21cut08:25:012
1207523994313,9cyclictest1286898-21ntp_states10:55:222
1207523994218,5cyclictest1308843-21diskstats12:00:182
1207523994215,10cyclictest1246050-21apt-get09:00:132
1207523994213,10cyclictest1292115-21memory11:10:202
1207523994127,9cyclictest1273240-21/usr/sbin/munin10:15:262
1207523994115,9cyclictest1242416-21apt-get08:50:142
1207523994113,9cyclictest1321192-21irqstats12:35:162
1207515994133,5cyclictest1291270-21kworker/u8:2+events_unbound11:25:201
1207528994032,4cyclictest1273929-21cron10:20:003
1207523994031,6cyclictest1214836-21sshd07:29:322
1207523994026,10cyclictest311ktimers/211:55:122
1207523994025,11cyclictest1253301-21apt-get09:20:142
1207523994024,11cyclictest1301033-21/usr/sbin/munin11:35:262
1207523994017,5cyclictest1269937-21users10:05:302
1207515994032,6cyclictest1264095-21kworker/u8:3+events_unbound10:00:031
1207515994027,5cyclictest1291270-21kworker/u8:2+events_unbound11:15:211
1207528993914,7cyclictest131rcu_preempt07:40:153
1207523993931,5cyclictest1280980-21munin-plugin-st10:40:002
1207523993931,4cyclictest1233898-21apt-get08:25:152
1207523993929,7cyclictest1208441-21/usr/sbin/munin07:10:252
1207523993927,9cyclictest1227634-21ntp_states08:05:222
1207523993925,11cyclictest1260121-21/usr/sbin/munin09:40:112
1207523993925,11cyclictest1258677-21/usr/sbin/munin09:35:202
1207523993921,13cyclictest1266229-21tail09:55:252
1207523993920,10cyclictest311ktimers/210:25:142
1207523993916,5cyclictest1297232-21meminfo11:25:202
1207523993915,13cyclictest630-21snmpd07:23:032
1207515993931,4cyclictest0-21swapper/107:30:011
1207523993831,3cyclictest1289958-21apt-get11:05:132
1207523993829,5cyclictest1244123-21apt-get08:55:142
1207523993827,7cyclictest1237638-21/usr/sbin/munin08:35:182
1207523993825,9cyclictest1235599-21apt-get08:30:162
1207523993825,10cyclictest1250115-21/usr/sbin/munin09:10:212
1207523993823,12cyclictest1225891-21/usr/sbin/munin08:00:232
1207523993823,11cyclictest1315294-21apt-get12:20:042
1207523993822,12cyclictest1299035-21sshd11:30:222
1207523993822,12cyclictest1251817-21/usr/sbin/munin09:15:222
1207523993822,10cyclictest1271801-21sed10:10:272
1207523993819,15cyclictest1267346-21fschecks_time10:00:172
1207515993820,6cyclictest1274189-21kworker/u8:3+events_unbound10:40:171
1207515993820,6cyclictest1274189-21kworker/u8:3+events_unbound10:40:161
1207513993830,4cyclictest1218300-21cron07:40:020
1207513993828,6cyclictest0-21swapper/012:20:000
1207513993828,6cyclictest0-21swapper/011:30:260
1207528993711,5cyclictest131rcu_preempt10:20:163
1207523993730,4cyclictest1209999-21apt-get07:15:152
1207523993730,4cyclictest1209999-21apt-get07:15:142
1207523993729,5cyclictest1274200-21apt-get10:20:172
1207523993729,4cyclictest0-21swapper/207:40:012
1207523993727,7cyclictest1249018-21sshd09:07:482
1207523993726,8cyclictest1256124-21cron09:26:002
1207523993726,8cyclictest1218545-21/usr/sbin/munin07:40:142
1207523993726,7cyclictest1319494-21/usr/sbin/munin12:30:192
1207523993725,9cyclictest1302468-21/usr/sbin/munin11:40:212
1207523993725,8cyclictest0-21swapper/212:25:122
1207523993724,8cyclictest1313934-21/usr/sbin/munin12:15:142
1207523993723,11cyclictest1264294-21/usr/sbin/munin09:50:242
1207523993723,11cyclictest1224335-21/usr/sbin/munin07:55:272
1207523993722,11cyclictest1293961-21/usr/sbin/munin11:15:222
1207515993729,4cyclictest0-21swapper/108:05:011
1207513993729,4cyclictest1270930-21fschecks_count10:10:150
1207528993616,13cyclictest1207511-21cyclictest10:57:113
1207528993613,8cyclictest40-21ksoftirqd/309:55:003
1207523993625,8cyclictest1229047-21/usr/sbin/munin08:10:192
1207523993625,7cyclictest1312242-21diskstats12:10:152
1207523993624,9cyclictest1303762-21/usr/sbin/munin11:45:162
1207523993624,9cyclictest1231549-21users08:15:282
1207523993623,9cyclictest1222075-21/usr/sbin/munin07:50:202
1207523993612,8cyclictest382-21ntpd11:00:242
1207523993612,8cyclictest382-21ntpd11:00:232
1207523993610,9cyclictest1256702-21apt-get09:30:132
1207513993626,6cyclictest0-21swapper/010:00:010
1207528993528,4cyclictest0-21swapper/312:00:273
1207528993515,12cyclictest131rcu_preempt11:55:203
1207528993514,12cyclictest40-21ksoftirqd/310:05:293
1207528993513,5cyclictest131rcu_preempt09:20:123
1207528993511,14cyclictest131rcu_preempt11:50:283
1207523993525,7cyclictest1216311-21sshd07:31:482
1207523993523,9cyclictest1261821-21/usr/sbin/munin09:45:192
1207523993521,9cyclictest1306311-21sed11:50:272
1207523993521,11cyclictest1310758-21/usr/sbin/munin12:05:182
1207523993521,11cyclictest1310758-21/usr/sbin/munin12:05:172
1207523993510,11cyclictest637-21runrttasks10:44:052
1207523993510,11cyclictest637-21runrttasks10:44:052
1207515993529,4cyclictest1252819-21kworker/u8:2+events_unbound09:35:081
1207515993528,4cyclictest1313588-21cron12:15:001
1207515993527,4cyclictest0-21swapper/109:40:241
1207515993524,7cyclictest0-21swapper/110:20:011
1207513993529,4cyclictest0-21swapper/010:55:130
1207513993528,4cyclictest0-21swapper/008:40:260
1207513993526,4cyclictest0-21swapper/008:24:410
1207513993525,7cyclictest1236641-21grep08:30:300
1207513993525,6cyclictest0-21swapper/011:15:260
1207528993423,7cyclictest0-21swapper/307:15:013
1207528993415,5cyclictest131rcu_preempt11:40:253
1207528993413,12cyclictest131rcu_preempt07:50:283
1207523993424,7cyclictest1284768-21/usr/sbin/munin10:50:182
1207523993423,8cyclictest1239463-21meminfo08:40:202
120751599346,6cyclictest131rcu_preempt09:30:121
1207515993424,7cyclictest440-21cron11:04:591
1207515993424,7cyclictest440-21cron11:04:591
1207515993417,6cyclictest1229984-21kworker/u8:2+events_unbound08:35:151
1207513993427,4cyclictest1275651-21sh10:25:000
1207513993426,4cyclictest0-21swapper/011:25:010
1207513993424,6cyclictest0-21swapper/010:15:260
1207513993423,7cyclictest1224661-21grep07:55:290
1207513993410,6cyclictest11-21ksoftirqd/012:20:250
120752899339,11cyclictest40-21ksoftirqd/311:30:213
120752899338,7cyclictest40-21ksoftirqd/307:40:023
120752899335,6cyclictest40-21ksoftirqd/309:25:253
1207528993324,6cyclictest1250689-21unixbench_multi09:10:273
1207528993323,6cyclictest1221573-21kworker/u8:2+events_unbound08:03:433
1207528993319,5cyclictest131rcu_preempt08:00:003
1207528993318,7cyclictest131rcu_preempt08:35:223
1207528993314,11cyclictest131rcu_preempt07:25:203
1207528993313,12cyclictest131rcu_preempt08:15:213
1207528993312,13cyclictest131rcu_preempt10:30:223
1207515993324,5cyclictest0-21swapper/110:40:001
1207513993327,4cyclictest0-21swapper/010:00:140
1207513993323,6cyclictest0-21swapper/008:10:270
1207513993322,7cyclictest1254284-21grep09:20:250
120751399331,17cyclictest11-21ksoftirqd/009:15:170
120752899328,7cyclictest0-21swapper/312:30:133
120752899328,15cyclictest40-21ksoftirqd/312:10:263
120752899328,13cyclictest40-21ksoftirqd/310:00:203
1207528993223,6cyclictest0-21swapper/308:10:273
1207528993219,4cyclictest131rcu_preempt08:32:513
1207528993213,6cyclictest131rcu_preempt08:20:263
1207528993213,11cyclictest131rcu_preempt07:30:223
1207528993210,4cyclictest0-21swapper/310:54:593
1207528993210,12cyclictest131rcu_preempt08:10:023
120751599325,5cyclictest131rcu_preempt09:25:131
1207515993224,4cyclictest0-21swapper/111:30:261
1207515993224,4cyclictest0-21swapper/109:10:001
120751399329,5cyclictest11-21ksoftirqd/008:50:010
1207513993222,6cyclictest0-21swapper/007:50:280
1207528993125,4cyclictest1235350-21kworker/u8:1+events_unbound08:44:473
1207528993118,11cyclictest1207511-21cyclictest10:45:143
120752899311,7cyclictest0-21swapper/311:00:153
120752899311,7cyclictest0-21swapper/311:00:153
1207528993116,6cyclictest131rcu_preempt12:15:143
1207528993114,5cyclictest131rcu_preempt11:20:173
1207528993112,12cyclictest131rcu_preempt10:25:273
1207528993111,12cyclictest131rcu_preempt09:05:203
1207528993110,6cyclictest40-21ksoftirqd/310:45:003
1207528993110,6cyclictest40-21ksoftirqd/310:44:593
120751599312,5cyclictest131rcu_preempt11:20:121
1207515993124,5cyclictest1233341-21kworker/u8:0+events_unbound08:29:441
1207515993124,4cyclictest1224647-21unixbench_multi07:55:301
1207515993123,4cyclictest0-21swapper/110:10:001
1207515993121,6cyclictest0-21swapper/109:20:021
120751599311,5cyclictest131rcu_preempt12:25:111
120751399318,5cyclictest11-21ksoftirqd/009:30:260
120751399317,5cyclictest11-21ksoftirqd/010:55:010
120751399317,5cyclictest11-21ksoftirqd/007:25:000
1207513993122,5cyclictest0-21swapper/012:30:000
1207513993121,7cyclictest1291417-21cron11:09:590
1207513993112,12cyclictest131rcu_preempt11:00:170
1207513993112,12cyclictest131rcu_preempt11:00:160
120752899302,7cyclictest40-21ksoftirqd/312:10:003
120752899302,7cyclictest40-21ksoftirqd/312:10:003
1207528993024,4cyclictest1278775-21kworker/u8:1+events_unbound10:39:423
1207528993023,4cyclictest0-21swapper/309:00:183
1207528993022,4cyclictest0-21swapper/309:45:083
1207528993020,6cyclictest0-21swapper/311:15:253
1207528993020,5cyclictest0-21swapper/309:40:253
120751599302,4cyclictest131rcu_preempt10:20:171
1207515993022,4cyclictest0-21swapper/107:30:271
120751399308,6cyclictest11-21ksoftirqd/010:10:010
120751399308,4cyclictest11-21ksoftirqd/010:35:010
120751399307,6cyclictest11-21ksoftirqd/010:35:030
120751399306,13cyclictest11-21ksoftirqd/011:12:340
120751399305,8cyclictest1300371-21fschecks_time11:35:160
120751399304,3cyclictest1296786-21dpkg11:25:130
1207513993023,3cyclictest0-21swapper/012:00:000
1207513993016,5cyclictest131rcu_preempt11:42:590
1207528992923,4cyclictest1316909-21kworker/u8:0+events_unbound12:21:283
1207528992921,4cyclictest0-21swapper/311:35:253
1207528992921,4cyclictest0-21swapper/308:50:013
1207528992916,5cyclictest131rcu_preempt07:20:163
1207528992913,5cyclictest131rcu_preempt11:25:233
1207528992911,5cyclictest131rcu_preempt07:15:053
1207528992911,5cyclictest131rcu_preempt07:15:053
1207528992911,11cyclictest131rcu_preempt09:35:213
1207528992910,10cyclictest131rcu_preempt10:10:033
120751599296,6cyclictest1241877-21kworker/u8:0+events_unbound08:55:021
120751599293,4cyclictest131rcu_preempt07:15:141
120751599293,4cyclictest131rcu_preempt07:15:141
1207515992922,4cyclictest0-21swapper/112:19:591
1207515992921,4cyclictest0-21swapper/111:55:151
120751399298,6cyclictest11-21ksoftirqd/008:05:250
120751399298,6cyclictest0-21swapper/010:45:280
120751399297,5cyclictest0-21swapper/009:00:210
120751399296,4cyclictest11-21ksoftirqd/007:40:270
120751399295,5cyclictest11-21ksoftirqd/011:50:260
120751399294,5cyclictest1213473-21cpu07:25:150
120751399292,9cyclictest1208526-21date07:10:270
1207513992920,6cyclictest0-21swapper/008:20:000
1207513992919,6cyclictest0-21swapper/009:40:240
1207513992915,4cyclictest131rcu_preempt09:35:240
1207513992911,6cyclictest0-21swapper/010:40:270
1207513992911,6cyclictest0-21swapper/010:40:270
1207528992821,4cyclictest0-21swapper/309:55:253
1207528992821,3cyclictest0-21swapper/312:25:243
1207528992819,6cyclictest0-21swapper/309:20:003
120752899281,13cyclictest0-21swapper/308:50:133
1207515992822,4cyclictest1245373-21kworker/u8:1+events_unbound09:14:521
1207515992820,4cyclictest0-21swapper/110:10:261
1207515992819,6cyclictest1219070-21grep07:40:221
120751399287,6cyclictest0-21swapper/012:30:160
120751399285,5cyclictest1304105-21missed_timers11:45:210
1207513992822,4cyclictest1310194-21cron12:04:590
1207513992819,6cyclictest0-21swapper/009:00:020
1207528992721,3cyclictest0-21swapper/311:45:263
1207528992717,3cyclictest131rcu_preempt09:30:263
1207515992721,3cyclictest0-21swapper/109:00:021
1207515992720,5cyclictest1252819-21kworker/u8:2+events_unbound09:48:051
1207515992719,5cyclictest0-21swapper/112:00:271
1207515992719,4cyclictest0-21swapper/107:50:291
120751599271,4cyclictest131rcu_preempt11:05:131
120751599271,3cyclictest131rcu_preempt07:20:161
1207515992712,3cyclictest24-21ksoftirqd/108:15:241
120751399271,16cyclictest0-21swapper/008:50:250
120751399271,15cyclictest0-21swapper/008:25:230
1207513992710,4cyclictest131rcu_preempt08:37:590
120751399270,11cyclictest1215339-21fschecks_time07:30:180
1207528992619,5cyclictest1233341-21kworker/u8:0+events_unbound08:26:073
120751599267,7cyclictest1291270-21kworker/u8:2+events_unbound11:37:301
1207515992620,4cyclictest1263258-21kworker/u8:0+events_unbound09:55:121
1207515992620,4cyclictest1252819-21kworker/u8:2+events_unbound09:23:261
120751599260,4cyclictest131rcu_preempt12:35:121
120751399264,4cyclictest11-21ksoftirqd/012:10:250
120751399263,5cyclictest1322521-21date12:40:000
120751399262,6cyclictest11-21ksoftirqd/009:05:130
1207513992618,4cyclictest0-21swapper/009:25:270
1207513992617,6cyclictest0-21swapper/009:50:140
120751399261,6cyclictest1311500-21taskset12:05:270
120751399261,6cyclictest1311500-21taskset12:05:260
120751399261,6cyclictest1276141-21basename10:25:190
1207515992517,5cyclictest0-21swapper/110:50:281
1207515992516,7cyclictest1286887-21ntp_offset10:55:211
120751599251,5cyclictest131rcu_preempt09:00:141
1207515992515,6cyclictest0-21swapper/111:10:171
120751399259,8cyclictest131rcu_preempt07:49:200
1207515992418,4cyclictest1207511-21cyclictest10:30:091
1207515992417,5cyclictest1226465-21kworker/u8:1+events_unbound08:05:251
1207515992413,8cyclictest1233055-21sed08:20:281
1207513992412,4cyclictest1249809-21fschecks_count09:10:170
120751599238,4cyclictest221rcuc/112:30:181
120751599236,5cyclictest1316909-21kworker/u8:0+events_unbound12:23:281
1207515992317,4cyclictest1282918-21kworker/u8:1+events_unbound10:46:561
1207515992317,3cyclictest0-21swapper/107:45:221
1207515992316,5cyclictest1235350-21kworker/u8:1+events_unbound08:45:251
1207515992316,4cyclictest0-21swapper/107:40:011
1207515992315,5cyclictest0-21swapper/108:10:171
120751599231,3cyclictest131rcu_preempt07:15:011
1207515992216,3cyclictest231ktimers/111:40:131
120751599221,11cyclictest0-21swapper/111:51:471
1207515991913,4cyclictest1264073-21ntp_offset09:50:201
1207515991813,3cyclictest1239012-21/usr/sbin/munin08:40:181
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional