You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-02 - 15:23
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackfslot4.osadl.org (updated Tue Dec 02, 2025 00:45:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1883322810,2sleep20-21swapper/219:05:212
2007822700,1sleep35064-21kworker/u8:2019:10:013
1928122560,1sleep112374-21kworker/u8:3719:05:291
2011698157145,9cyclictest10-21ksoftirqd/019:47:100
2011698152143,5cyclictest118150irq/29-4f00000.22:12:340
2011898150143,5cyclictest29623-21kworker/u8:8019:47:091
2011898149142,4cyclictest29643-21kworker/u8:522:12:341
2011998136130,4cyclictest29558-21kworker/u8:1219:47:092
2011998135129,4cyclictest29690-21kworker/u8:8022:12:342
31021260,1sleep1621-21kswapd000:30:211
31021260,1sleep1621-21kswapd000:30:201
3242921220,3sleep20-21swapper/223:20:432
2011698117107,7cyclictest10-21ksoftirqd/021:17:160
1647821120,2sleep216475-21ntpq21:40:332
102552980,2sleep20-21swapper/223:43:112
20118989687,5cyclictest26982-21kworker/u8:3822:07:271
20118989687,5cyclictest26982-21kworker/u8:3822:07:271
20116989689,3cyclictest1050ksoftirqd/022:07:270
20116989689,3cyclictest1050ksoftirqd/022:07:270
20116988781,3cyclictest1050ksoftirqd/023:57:000
20118988679,4cyclictest12386-21kworker/u8:1223:57:011
20119987567,4cyclictest14779-21kworker/u8:3823:57:012
20118986253,5cyclictest5385-21kworker/u8:2821:22:241
20118986253,5cyclictest5385-21kworker/u8:2821:22:241
20116986155,3cyclictest1050ksoftirqd/021:22:240
20116986155,3cyclictest1050ksoftirqd/021:22:240
210962580,4chrt0-21swapper/120:35:361
20116985448,3cyclictest1050ksoftirqd/021:32:380
20116985444,7cyclictest10-21ksoftirqd/022:51:280
20116985338,12cyclictest10-21ksoftirqd/023:11:580
244662490,2sleep124465-21df23:05:211
20118984841,4cyclictest16103-21kworker/u8:2422:51:281
93692460,3sleep10-21swapper/121:25:321
20119984640,3cyclictest5392-21kworker/u8:3621:22:242
20119984640,3cyclictest5392-21kworker/u8:3621:22:242
63142450,2sleep36315-21cat19:55:463
2012298451,20cyclictest26683-21ntpq20:50:383
2012298443,21cyclictest26976-21kworker/u8:3222:05:353
2012298443,21cyclictest26976-21kworker/u8:3222:05:343
2012298441,23cyclictest16508-21ntpq21:40:353
20116984429,8cyclictest10-21ksoftirqd/022:25:170
181202440,2sleep318115-21modprobe20:29:593
120442440,3sleep30-21swapper/321:30:453
2012298431,23cyclictest13136-21ntpq22:40:333
2012298431,22cyclictest4583-21ntpq23:30:303
20116984337,3cyclictest1050ksoftirqd/019:57:240
20116984333,7cyclictest10-21ksoftirqd/019:27:420
290112420,2sleep127016-21kworker/u8:4500:20:441
290112420,2sleep127016-21kworker/u8:4500:20:441
2012298421,21cyclictest15471-21ntpq20:20:323
2012298421,21cyclictest11835-21ntpq20:10:503
2012298421,19cyclictest11296-21ssh22:36:133
20118984236,3cyclictest16106-21kworker/u8:2723:11:571
20118984235,4cyclictest29627-21kworker/u8:8419:57:231
2012298412,19cyclictest219452kworker/u8:2519:15:283
2012298411,23cyclictest8799-21forks23:40:223
2012298411,21cyclictest15480-21ntpq22:45:293
2012298411,20cyclictest3612-21ntpq00:35:343
2012298411,19cyclictest9773-21ntpq20:05:433
2012298411,18cyclictest14071-21ntpq23:50:333
190162410,5sleep3311ksoftirqd/322:54:273
2012298401,21cyclictest27163-21ld19:25:343
2012298401,21cyclictest20557-21seq19:10:183
2012298401,20cyclictest21296-21ntpq00:05:383
2012298401,18cyclictest3597-21ntpq22:20:283
2012298401,18cyclictest18626-21diskstats20:30:213
20119984033,4cyclictest7787-21kworker/u8:1821:32:382
44922390,3sleep22119-21kworker/u8:2200:37:492
2012298391,20cyclictest18829-21ntpq00:00:313
2012298391,19cyclictest2301-21ntpq23:25:363
2012298391,19cyclictest13484-21memory20:15:293
2012298381,17cyclictest27479-21ntpq23:10:343
2012298371,19cyclictest23896-21ntp_states21:55:423
2012298371,18cyclictest32662-21ntpq21:05:473
2012298371,18cyclictest29790-21ntpq23:15:323
2012298371,17cyclictest2791-21taskset19:46:553
2012298371,16cyclictest30606-21ntp_states21:00:353
2012298371,16cyclictest1139-21ntpq22:15:293
2011998375,7cyclictest121rcu_preempt20:45:162
20119983729,5cyclictest16093-21kworker/u8:1422:51:282
20118983731,4cyclictest23805-21kworker/u8:419:27:431
2012298361,20cyclictest7834-21runrttasks23:37:553
2012298361,20cyclictest7834-21runrttasks23:37:543
2012298361,19cyclictest8358-21ntpq22:30:303
2012298361,19cyclictest8358-21ntpq22:30:303
2012298361,17cyclictest7077-21ntp_states21:20:343
2012298361,17cyclictest7077-21ntp_states21:20:343
2012298361,17cyclictest27685-21cp20:55:033
2012298361,16cyclictest25657-21ssh23:06:313
2012298361,15cyclictest23704-21latency_hist20:45:023
2012298361,15cyclictest23704-21latency_hist20:45:013
20116983629,4cyclictest1050ksoftirqd/020:32:130
311172350,3sleep10-21swapper/100:25:471
230352350,6sleep3311ksoftirqd/300:10:323
230352350,6sleep3311ksoftirqd/300:10:313
20122983516,14cyclictest10950-21/usr/sbin/munin23:45:163
2012298351,29cyclictest19571-21/usr/sbin/munin22:55:193
2012298351,29cyclictest19571-21/usr/sbin/munin22:55:193
2012298351,19cyclictest22561-21runrttasks21:54:183
2012298351,17cyclictest595-21df_inode00:30:353
2012298351,17cyclictest595-21df_inode00:30:343
20118983527,5cyclictest17868-21kworker/u8:720:32:131
20118983517,7cyclictest19-21ksoftirqd/122:20:171
2012298341,29cyclictest30958-21ntpq00:25:343
2012298341,19cyclictest19593-21kworker/u8:900:15:173
320002330,8sleep1191ksoftirqd/123:20:251
2012298332,14cyclictest77922kworker/u8:2321:27:073
20122983314,16cyclictest28478-21/usr/sbin/munin19:30:223
2012298331,16cyclictest15907-21hddtemp_smartct23:55:233
20118983316,4cyclictest19-21ksoftirqd/119:40:171
20118983314,3cyclictest19-21ksoftirqd/120:00:331
20116983323,7cyclictest10-21ksoftirqd/020:56:470
94992320,14sleep2121rcu_preempt22:34:192
94992320,14sleep2121rcu_preempt22:34:182
271232320,12sleep225-21ksoftirqd/220:51:512
232932320,7sleep3311ksoftirqd/323:01:333
2012298325,24cyclictest1101-21grep23:25:013
2012298322,26cyclictest20117-21latency_hist21:50:013
2012298321,27cyclictest3570-21/usr/sbin/munin19:50:213
2012298321,27cyclictest14410-21sed21:35:423
2012298321,24cyclictest258432kworker/u8:2219:40:173
2012298321,16cyclictest20217-21/usr/sbin/munin20:35:243
2012298321,13cyclictest5457-21hddtemp_smartct22:25:233
20118983216,8cyclictest19-21ksoftirqd/120:40:381
20118983216,8cyclictest19-21ksoftirqd/120:40:381
20118983215,4cyclictest19-21ksoftirqd/123:40:411
20118983215,4cyclictest19-21ksoftirqd/122:15:361
20118983215,4cyclictest19-21ksoftirqd/121:30:331
20118983214,4cyclictest19-21ksoftirqd/121:40:311
20118983214,4cyclictest19-21ksoftirqd/121:10:361
20118983213,4cyclictest19-21ksoftirqd/121:00:281
20118983213,4cyclictest19-21ksoftirqd/120:45:341
20116983226,3cyclictest1050ksoftirqd/020:51:530
2012298311,27cyclictest7032-21/usr/sbin/munin20:00:473
2012298311,26cyclictest30902-21ntpq19:35:323
2012298311,26cyclictest30902-21ntpq19:35:313
2012298311,26cyclictest29875-21munin-run00:25:003
2012298311,26cyclictest29875-21munin-run00:24:593
2012298311,26cyclictest24192-21df20:45:203
2012298311,15cyclictest0-21swapper/322:13:173
20119983123,5cyclictest16112-21kworker/u8:3323:11:582
20118983124,4cyclictest23401-21kworker/u8:1120:51:531
20118983117,4cyclictest19-21ksoftirqd/100:10:461
20118983117,4cyclictest19-21ksoftirqd/100:10:451
20118983117,3cyclictest19-21ksoftirqd/121:50:131
20118983116,5cyclictest19-21ksoftirqd/122:45:341
20118983115,4cyclictest19-21ksoftirqd/123:25:181
20118983114,5cyclictest19-21ksoftirqd/121:35:141
20118983114,4cyclictest19-21ksoftirqd/123:35:311
20118983114,4cyclictest19-21ksoftirqd/123:35:301
20118983114,4cyclictest19-21ksoftirqd/120:05:341
20118983114,4cyclictest19-21ksoftirqd/100:00:381
20118983114,3cyclictest19-21ksoftirqd/121:45:421
20118983113,4cyclictest19-21ksoftirqd/122:40:271
20118983113,4cyclictest19-21ksoftirqd/122:30:321
20118983113,4cyclictest19-21ksoftirqd/122:30:321
20118983113,4cyclictest19-21ksoftirqd/119:35:321
20118983113,4cyclictest19-21ksoftirqd/119:35:311
20118983112,4cyclictest19-21ksoftirqd/123:00:011
20118983112,4cyclictest19-21ksoftirqd/123:00:011
20118983112,3cyclictest19-21ksoftirqd/121:15:141
180612310,2sleep20-21swapper/200:00:162
155372310,3sleep115534-21ntp_states20:20:371
35602300,17sleep129568-21kworker/u8:2419:50:201
285062300,3sleep228505-21sed00:20:282
285062300,3sleep228505-21sed00:20:282
2012298303,23cyclictest5897-21munin-run21:20:003
20119983022,6cyclictest4684-21kworker/u8:1219:57:232
2011998301,4cyclictest121rcu_preempt23:25:252
2011898306,4cyclictest121rcu_preempt23:00:381
20118983016,4cyclictest19-21ksoftirqd/119:15:331
20118983015,4cyclictest19-21ksoftirqd/122:25:391
20118983015,4cyclictest19-21ksoftirqd/120:15:361
20118983015,3cyclictest19-21ksoftirqd/100:05:141
20118983014,9cyclictest19-21ksoftirqd/123:17:021
20118983014,4cyclictest19-21ksoftirqd/100:15:251
20118983014,10cyclictest19-21ksoftirqd/119:10:531
20118983013,4cyclictest19-21ksoftirqd/100:35:191
20118983013,3cyclictest19-21ksoftirqd/122:00:191
20118983012,4cyclictest19-21ksoftirqd/123:45:281
20118983012,4cyclictest19-21ksoftirqd/123:30:331
20118983011,4cyclictest19-21ksoftirqd/120:10:451
35522290,2sleep20-21swapper/222:20:272
309882290,2sleep20-21swapper/221:00:442
228952290,2sleep20-21swapper/220:40:332
228952290,2sleep20-21swapper/220:40:332
219812290,6sleep220111-21cyclictest19:12:492
2012298292,23cyclictest2170-21ntpq21:10:343
2012298291,24cyclictest24309-21modprobe19:20:023
2011998298,5cyclictest121rcu_preempt23:35:442
2011998298,5cyclictest121rcu_preempt23:35:432
2011998297,11cyclictest25-21ksoftirqd/220:10:352
20118982914,4cyclictest19-21ksoftirqd/123:50:221
20118982914,4cyclictest19-21ksoftirqd/121:05:221
20118982914,4cyclictest19-21ksoftirqd/120:25:411
20118982914,3cyclictest19-21ksoftirqd/119:20:431
20118982912,4cyclictest19-21ksoftirqd/121:55:241
20118982911,8cyclictest19-21ksoftirqd/119:30:331
20118982911,3cyclictest19-21ksoftirqd/122:36:371
20116982924,3cyclictest1050ksoftirqd/020:41:300
20116982924,3cyclictest1050ksoftirqd/020:41:300
316322280,7sleep2251ksoftirqd/219:39:002
316322280,7sleep2251ksoftirqd/219:38:592
2012298280,25cyclictest24548-21kworker/u8:722:02:143
2011698289,17cyclictest111rcuc/023:05:390
2011698283,22cyclictest111rcuc/021:40:460
20116982820,5cyclictest10-21ksoftirqd/000:00:140
293962270,2sleep20-21swapper/220:59:112
233222270,2sleep221091-21kworker/u8:723:01:442
2011998277,3cyclictest25-21ksoftirqd/221:10:202
2011998271,3cyclictest121rcu_preempt23:05:112
20119982713,4cyclictest25-21ksoftirqd/223:45:402
20119982711,4cyclictest25-21ksoftirqd/200:25:402
2011998271,14cyclictest0-21swapper/220:00:202
20118982713,3cyclictest19-21ksoftirqd/120:55:461
2011698275,15cyclictest121rcu_preempt21:10:260
253512260,3sleep225346-21apt-get00:15:142
2011998269,3cyclictest17723-21sed20:25:472
2011998268,9cyclictest25-21ksoftirqd/219:50:292
2011998266,10cyclictest25917-21irqrtprio22:00:252
2011998263,8cyclictest121rcu_preempt00:10:522
2011998263,8cyclictest121rcu_preempt00:10:512
2011998262,3cyclictest121rcu_preempt21:35:282
2011998261,9cyclictest121rcu_preempt22:35:252
2011998261,9cyclictest121rcu_preempt21:50:402
2011998261,8cyclictest121rcu_preempt22:25:422
2011998261,6cyclictest21202-21/usr/sbin/munin00:05:302
2011998261,3cyclictest121rcu_preempt23:15:262
2011998261,14cyclictest0-21swapper/219:25:262
20119982611,11cyclictest25-21ksoftirqd/221:45:172
2011698269,6cyclictest111rcuc/000:30:330
2011698269,6cyclictest111rcuc/000:30:320
2011698262,22cyclictest111rcuc/020:15:390
20116982612,4cyclictest118150irq/29-4f00000.19:15:260
2011698261,11cyclictest569-21/usr/sbin/munin22:15:210
20116982610,13cyclictest111rcuc/022:40:200
160082260,1sleep213895-21kworker/u8:822:46:012
56802250,1sleep25367-21kworker/u8:921:18:572
2011998256,10cyclictest25-21ksoftirqd/222:15:402
2011998253,3cyclictest121rcu_preempt22:05:592
2011998253,3cyclictest121rcu_preempt22:05:592
2011998252,3cyclictest121rcu_preempt19:20:012
2011998251,9cyclictest121rcu_preempt21:55:542
2011998251,3cyclictest121rcu_preempt22:40:312
2011998251,2cyclictest121rcu_preempt19:40:152
20119982512,3cyclictest25-21ksoftirqd/219:25:022
2011998251,15cyclictest15330-21cron23:55:012
2011998251,13cyclictest0-21swapper/220:20:452
2011698251,22cyclictest111rcuc/022:35:410
20116982511,4cyclictest118150irq/29-4f00000.22:55:260
20116982511,4cyclictest118150irq/29-4f00000.22:55:250
20116982511,4cyclictest118150irq/29-4f00000.19:20:460
20116982510,4cyclictest118150irq/29-4f00000.19:40:310
2011998245,3cyclictest25-21ksoftirqd/200:30:122
2011998245,3cyclictest25-21ksoftirqd/200:30:112
2011998243,7cyclictest121rcu_preempt20:05:482
2011998243,6cyclictest121rcu_preempt21:25:192
2011998243,3cyclictest121rcu_preempt21:05:322
2011998243,2cyclictest121rcu_preempt20:15:422
2011998242,4cyclictest121rcu_preempt23:30:292
2011998241,4cyclictest121rcu_preempt22:55:312
2011998241,4cyclictest121rcu_preempt22:55:312
20119982411,3cyclictest25-21ksoftirqd/219:30:262
2011698241,20cyclictest111rcuc/023:35:290
2011698241,20cyclictest111rcuc/023:35:290
2011698241,19cyclictest111rcuc/000:05:090
2011998232,3cyclictest121rcu_preempt20:35:212
2011998231,3cyclictest121rcu_preempt20:30:142
2011698239,4cyclictest118150irq/29-4f00000.21:45:240
2011698239,3cyclictest118150irq/29-4f00000.20:00:370
2011698232,18cyclictest111rcuc/022:20:360
20116982316,4cyclictest10-21ksoftirqd/023:40:160
20116982311,3cyclictest118150irq/29-4f00000.19:10:350
2011698228,11cyclictest118150irq/29-4f00000.23:20:320
2011698228,11cyclictest111rcuc/000:25:390
2011698227,13cyclictest111rcuc/023:30:310
2011698225,15cyclictest111rcuc/019:50:320
2011698222,18cyclictest111rcuc/022:45:290
20116982217,3cyclictest118150irq/29-4f00000.21:00:400
20116982217,3cyclictest10-21ksoftirqd/022:30:290
20116982217,3cyclictest10-21ksoftirqd/022:30:290
20116982217,3cyclictest10-21ksoftirqd/021:05:330
20116982217,3cyclictest10-21ksoftirqd/020:10:480
20116982216,4cyclictest10-21ksoftirqd/023:50:150
20116982216,4cyclictest10-21ksoftirqd/023:45:140
20116982216,4cyclictest10-21ksoftirqd/023:15:300
20116982216,4cyclictest10-21ksoftirqd/021:55:210
20116982216,4cyclictest10-21ksoftirqd/021:25:180
20116982216,4cyclictest10-21ksoftirqd/020:35:160
20116982216,3cyclictest111rcuc/019:30:350
20116982216,3cyclictest10-21ksoftirqd/019:35:330
20116982216,3cyclictest10-21ksoftirqd/019:35:330
20116982215,4cyclictest10-21ksoftirqd/023:25:150
20116982213,7cyclictest10-21ksoftirqd/000:20:130
20116982213,7cyclictest10-21ksoftirqd/000:20:130
20116982212,8cyclictest10-21ksoftirqd/000:35:190
2011698221,19cyclictest111rcuc/023:00:220
20116982211,9cyclictest111rcuc/020:45:310
20116982211,9cyclictest111rcuc/020:20:190
2011698219,10cyclictest111rcuc/000:15:110
20116982116,3cyclictest10-21ksoftirqd/022:00:390
20116982115,4cyclictest10-21ksoftirqd/021:50:300
20116982115,4cyclictest10-21ksoftirqd/021:35:150
20116982115,4cyclictest10-21ksoftirqd/020:25:190
20116982114,5cyclictest10-21ksoftirqd/020:05:330
20116982114,4cyclictest10-21ksoftirqd/000:10:310
20116982114,4cyclictest10-21ksoftirqd/000:10:310
20116981610,3cyclictest10-21ksoftirqd/019:10:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional