You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-17 - 15:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Mon Nov 17, 2025 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3263999250,24cyclictest9335-21cron23:55:003
3262399250,24cyclictest27626-21ls22:20:000
3262399240,23cyclictest0-21swapper/022:11:440
32639992323,0cyclictest6715-21apt-get20:31:453
32639992323,0cyclictest0-21swapper/300:16:473
32639992320,2cyclictest17994-21cron22:00:003
32632992321,1cyclictest8736-21sleep20:35:012
32628992321,1cyclictest29050-21apt-get00:36:441
32628992320,2cyclictest313-21systemd-journal23:30:001
3262899230,22cyclictest8806-21apt-get21:41:451
3262899230,22cyclictest16342-21munin-run20:50:001
3262399230,2cyclictest11748-21find00:00:000
3262399230,22cyclictest593-21dbus-daemon21:05:000
3262399230,22cyclictest20064-21cron23:10:000
32639992219,2cyclictest3609-21munin-run21:30:003
3263999220,22cyclictest0-21swapper/320:06:433
3263999220,21cyclictest8806-21apt-get21:41:453
3263999220,21cyclictest17440-21apt-get19:46:423
3263999220,21cyclictest10231-21apt-get19:31:443
3263999220,21cyclictest0-21swapper/321:46:453
3263999220,21cyclictest0-21swapper/319:45:013
3263999220,21cyclictest0-21swapper/300:36:443
3263999220,21cyclictest0-21swapper/300:26:443
3263999220,1cyclictest710-21apt-get21:25:003
32632992220,1cyclictest21836-21idleruntime-cro19:55:002
32632992219,2cyclictest7827-21apt-get19:26:422
32632992219,2cyclictest7827-21apt-get19:26:422
3263299220,21cyclictest3214-21apt-get22:36:452
3263299220,21cyclictest25213-21idleruntime-cro22:15:002
3263299220,21cyclictest21863-21apt-get00:21:432
3263299220,21cyclictest12235-21basename19:35:002
3263299220,21cyclictest0-21swapper/222:41:432
3263299220,21cyclictest0-21swapper/219:36:422
32628992220,1cyclictest9369-21awk23:55:001
32628992220,1cyclictest11764-40mandb00:00:001
3262899220,21cyclictest31144-21apt-get21:21:421
3262899220,21cyclictest0-21swapper/120:41:431
3262899220,1cyclictest4313-21apt-get20:26:431
3262899220,1cyclictest17046-21apt-get00:11:421
3262899220,1cyclictest16658-21perf00:10:001
32623992222,0cyclictest0-21swapper/020:26:420
32623992220,1cyclictest28345-21munin-plugin-st21:15:000
3262399220,21cyclictest19454-21perf19:50:000
3262399220,21cyclictest0-21swapper/022:01:410
3262399220,21cyclictest0-21swapper/020:36:450
3262399220,21cyclictest0-21swapper/020:16:420
3262399220,21cyclictest0-21swapper/019:16:420
3262399220,21cyclictest0-21swapper/019:11:420
3262399220,1cyclictest141rcuc/020:00:000
3262399220,1cyclictest13541-21perf20:45:000
32639992120,1cyclictest22871-21apt-get23:16:423
32639992120,0cyclictest0-21swapper/323:36:463
3263999210,21cyclictest0-21swapper/320:45:053
3263999210,20cyclictest20814-21apt-get22:06:463
3263999210,20cyclictest0-21swapper/323:26:443
3263999210,20cyclictest0-21swapper/322:40:013
3263999210,20cyclictest0-21swapper/321:01:423
3263999210,20cyclictest0-21swapper/320:51:423
3263999210,20cyclictest0-21swapper/319:36:423
3263999210,20cyclictest0-21swapper/300:31:433
3263999210,1cyclictest1419-21cron20:20:003
3263999210,1cyclictest0-21swapper/323:25:003
3263999210,1cyclictest0-21swapper/322:46:413
32632992118,2cyclictest13530-21grep20:44:592
3263299210,20cyclictest7451-21basename19:25:002
3263299210,1cyclictest9112-21apt-get20:36:452
3263299210,1cyclictest3988-21apt-get21:31:432
3263299210,0cyclictest7351-21apt-get23:51:432
3263299210,0cyclictest26330-21apt-get21:11:442
3263299210,0cyclictest19448-21apt-get00:16:472
3263299210,0cyclictest0-21swapper/223:41:452
3263299210,0cyclictest0-21swapper/220:51:432
3263299210,0cyclictest0-21swapper/200:26:442
32628992121,0cyclictest0-21swapper/121:46:531
32628992121,0cyclictest0-21swapper/120:51:431
32628992120,1cyclictest598-21rs:main1
32628992119,1cyclictest0-21swapper/122:00:001
3262899210,21cyclictest0-21swapper/121:31:441
3262899210,21cyclictest0-21swapper/119:26:411
3262899210,21cyclictest0-21swapper/119:26:411
3262899210,20cyclictest0-21swapper/122:51:421
3262899210,20cyclictest0-21swapper/121:36:431
3262899210,20cyclictest0-21swapper/120:06:431
3262899210,20cyclictest0-21swapper/119:51:411
3262899210,1cyclictest2428-21latency_hist22:35:001
3262899210,1cyclictest18413-21apt-get22:01:431
3262899210,0cyclictest6074-21apt-get22:41:431
3262899210,0cyclictest25293-21apt-get23:21:441
32623992121,0cyclictest0-21swapper/022:40:020
32623992120,0cyclictest0-21swapper/023:26:450
32623992120,0cyclictest0-21swapper/000:06:420
32623992119,1cyclictest22469-21cron23:15:000
3262399210,21cyclictest0-21swapper/023:21:440
3262399210,0cyclictest19835-21apt-get19:51:400
3262399210,0cyclictest0-21swapper/021:46:450
32639992020,0cyclictest0-21swapper/323:46:443
32639992020,0cyclictest0-21swapper/322:16:443
32639992020,0cyclictest0-21swapper/320:21:433
32639992019,0cyclictest18080-21apt-get23:06:433
32639992018,1cyclictest11731-21cron00:00:003
3263999200,0cyclictest0-21swapper/321:36:443
3263999200,0cyclictest0-21swapper/300:01:453
32632992020,0cyclictest24652-21apt-get20:01:432
32632992020,0cyclictest12258-21apt-get00:01:452
32632992020,0cyclictest0-21swapper/223:06:432
32632992020,0cyclictest0-21swapper/222:51:422
32632992020,0cyclictest0-21swapper/221:25:012
32632992020,0cyclictest0-21swapper/221:21:422
32632992020,0cyclictest0-21swapper/220:21:432
32632992019,1cyclictest13263-21apt22:56:422
3263299200,20cyclictest0-21swapper/222:26:452
3263299200,20cyclictest0-21swapper/219:41:442
3263299200,20cyclictest0-21swapper/219:11:422
3263299200,19cyclictest16652-21idleruntime-cro00:10:002
3263299200,0cyclictest30085-21apt-get23:31:432
3263299200,0cyclictest0-21swapper/223:11:422
3263299200,0cyclictest0-21swapper/200:11:432
32628992020,0cyclictest23209-21apt-get22:11:431
32628992020,0cyclictest0-21swapper/123:31:431
32628992020,0cyclictest0-21swapper/123:16:421
32628992020,0cyclictest0-21swapper/122:16:431
32628992020,0cyclictest0-21swapper/120:31:441
32628992020,0cyclictest0-21swapper/120:21:431
32628992020,0cyclictest0-21swapper/119:56:411
32628992020,0cyclictest0-21swapper/119:16:421
32628992018,1cyclictest24261-21apt-get00:26:431
32628992017,2cyclictest612-21gdbus22:30:001
3262899200,0cyclictest32490-21apt-get23:36:471
3262899200,0cyclictest0-21swapper/122:06:461
3262899200,0cyclictest0-21swapper/120:36:441
32623992020,0cyclictest30425-21apt-get22:26:460
32623992020,0cyclictest0-21swapper/020:21:420
32623992020,0cyclictest0-21swapper/000:21:430
32623992020,0cyclictest0-21swapper/000:16:460
32623992019,0cyclictest0-21swapper/022:56:410
32623992019,0cyclictest0-21swapper/000:36:440
32623992018,1cyclictest598-21rs:main0
32623992018,1cyclictest593-21dbus-daemon22:55:010
32639991919,0cyclictest0-21swapper/322:36:453
32639991919,0cyclictest0-21swapper/322:21:433
32639991919,0cyclictest0-21swapper/320:56:433
32639991919,0cyclictest0-21swapper/320:41:433
32639991919,0cyclictest0-21swapper/319:26:423
32639991919,0cyclictest0-21swapper/319:26:423
32639991919,0cyclictest0-21swapper/319:16:413
32639991918,1cyclictest2414-21sh22:35:003
32639991918,0cyclictest0-21swapper/321:51:463
32639991918,0cyclictest0-21swapper/321:51:453
32639991917,1cyclictest331rcuc/319:59:593
32639991917,1cyclictest30765-21munin-plugin-st21:20:013
32632991919,0cyclictest0-21swapper/223:36:472
32632991919,0cyclictest0-21swapper/223:21:432
32632991919,0cyclictest0-21swapper/222:46:422
32632991919,0cyclictest0-21swapper/222:31:432
32632991919,0cyclictest0-21swapper/200:36:432
32632991919,0cyclictest0-21swapper/200:31:422
32628991919,0cyclictest8469-21apt-get22:46:421
32628991919,0cyclictest26656-21apt-get00:31:431
32628991919,0cyclictest0-21swapper/123:46:441
32628991919,0cyclictest0-21swapper/121:51:461
32628991919,0cyclictest0-21swapper/121:51:461
32628991919,0cyclictest0-21swapper/120:11:451
32628991918,0cyclictest0-21swapper/120:01:431
32628991916,2cyclictest17707-21awk23:05:001
32623991919,0cyclictest0-21swapper/021:56:440
32623991919,0cyclictest0-21swapper/021:41:450
32623991919,0cyclictest0-21swapper/021:16:410
32623991919,0cyclictest0-21swapper/019:26:400
32623991919,0cyclictest0-21swapper/019:26:400
32623991918,0cyclictest0-21swapper/021:36:440
32623991917,1cyclictest13599-21apt-get21:51:460
32623991917,1cyclictest13599-21apt-get21:51:460
32623991917,1cyclictest0-21swapper/023:45:010
3262399190,18cyclictest15014-21/usr/sbin/munin19:41:520
32639991818,0cyclictest0-21swapper/323:31:433
32639991818,0cyclictest0-21swapper/320:01:423
32632991818,0cyclictest28024-21apt-get22:21:432
32632991818,0cyclictest0-21swapper/220:11:452
32632991817,1cyclictest9751-21apt-get23:56:432
32632991817,0cyclictest0-21swapper/221:51:462
32632991817,0cyclictest0-21swapper/221:51:462
32632991816,1cyclictest20420-21basename22:05:012
32628991818,0cyclictest0-21swapper/123:06:421
32628991818,0cyclictest0-21swapper/121:11:441
32628991818,0cyclictest0-21swapper/121:01:421
32628991818,0cyclictest0-21swapper/100:01:451
32628991816,1cyclictest21479-21munin-plugin-st00:20:001
3262899180,1cyclictest30713-21cron21:20:011
32623991818,0cyclictest0-21swapper/022:31:450
32623991817,0cyclictest0-21swapper/021:25:010
3262399180,17cyclictest5437-21apt-get19:21:450
32639991717,0cyclictest0-21swapper/322:11:443
32639991717,0cyclictest0-21swapper/321:11:443
32639991717,0cyclictest0-21swapper/319:51:393
32639991717,0cyclictest0-21swapper/300:06:433
3263999170,0cyclictest0-21swapper/319:11:413
32632991717,0cyclictest14317-21apt-get20:46:422
32632991717,0cyclictest0-21swapper/223:01:462
32632991717,0cyclictest0-21swapper/222:06:462
32632991717,0cyclictest0-21swapper/220:06:432
32632991715,1cyclictest21142-21perf21:00:002
32632991715,1cyclictest0-21swapper/220:30:012
3263299170,17cyclictest2958-21apt-get19:16:422
32628991717,0cyclictest2507-21apt-get23:41:451
32628991717,0cyclictest20474-21apt-get23:11:411
32628991717,0cyclictest0-21swapper/122:35:021
32628991717,0cyclictest0-21swapper/120:56:441
32628991717,0cyclictest0-21swapper/119:46:421
32628991715,1cyclictest30014-21awk22:25:001
32623991717,0cyclictest0-21swapper/023:01:440
32623991717,0cyclictest0-21swapper/021:31:440
32623991717,0cyclictest0-21swapper/019:31:440
32623991717,0cyclictest0-21swapper/000:26:440
32623991715,1cyclictest16715-21apt-get20:51:430
32623991715,1cyclictest0-21swapper/020:06:510
32639991616,0cyclictest0-21swapper/321:06:443
32639991616,0cyclictest0-21swapper/320:36:453
32639991616,0cyclictest0-21swapper/300:21:423
32639991615,1cyclictest4313-21apt-get20:26:433
32639991615,1cyclictest18413-21apt-get22:01:423
32639991614,1cyclictest0-21swapper/322:51:483
32632991616,0cyclictest0-21swapper/219:56:412
3263299160,1cyclictest10807-21awk21:44:592
3263299160,15cyclictest6971-21perf23:50:012
32628991616,0cyclictest0-21swapper/100:21:421
32623991616,0cyclictest0-21swapper/023:36:470
32623991616,0cyclictest0-21swapper/021:21:430
32623991616,0cyclictest0-21swapper/019:36:420
32623991616,0cyclictest0-21swapper/000:11:420
3262399161,14cyclictest0-21swapper/023:55:000
3262399161,10cyclictest0-21swapper/020:46:420
32632991515,0cyclictest0-21swapper/220:16:432
3263299150,8cyclictest0-21swapper/219:46:412
3263299150,14cyclictest30353-21unixbench_multi21:16:512
32628991515,0cyclictest0-21swapper/121:25:001
3262899150,14cyclictest0-21swapper/119:21:521
3262899150,0cyclictest0-21swapper/121:06:441
32623991515,0cyclictest0-21swapper/023:16:420
32623991515,0cyclictest0-21swapper/000:31:430
32623991514,0cyclictest0-21swapper/020:56:440
3262399150,12cyclictest0-21swapper/022:06:440
32639991414,0cyclictest0-21swapper/322:26:463
3263999140,14cyclictest0-21swapper/323:42:363
3263999140,14cyclictest0-21swapper/323:10:223
3263999140,14cyclictest0-21swapper/321:30:493
3263999140,14cyclictest0-21swapper/320:11:513
3263999140,14cyclictest0-21swapper/319:23:573
3263999140,14cyclictest0-21swapper/300:11:343
3263999140,0cyclictest0-21swapper/323:00:273
3263999140,0cyclictest0-21swapper/322:55:413
32632991414,0cyclictest0-21swapper/221:37:262
32632991414,0cyclictest0-21swapper/221:02:442
3263299140,14cyclictest0-21swapper/223:25:442
3263299140,0cyclictest0-21swapper/223:16:132
3263299140,0cyclictest0-21swapper/221:46:272
3263299140,0cyclictest0-21swapper/221:06:442
32628991414,0cyclictest0-21swapper/120:15:391
32628991414,0cyclictest0-21swapper/119:40:361
3262899140,14cyclictest0-21swapper/119:35:461
3262899140,0cyclictest0-21swapper/119:30:011
32623991414,0cyclictest0-21swapper/021:06:440
32623991414,0cyclictest0-21swapper/020:31:110
32623991414,0cyclictest0-21swapper/000:00:070
32623991413,1cyclictest0-21swapper/022:35:010
3262399140,14cyclictest0-21swapper/023:30:550
3262399140,14cyclictest0-21swapper/022:45:580
3262399140,0cyclictest0-21swapper/020:10:020
3262399140,0cyclictest0-21swapper/020:00:500
32632991313,0cyclictest0-21swapper/222:15:062
32623991313,0cyclictest0-21swapper/022:20:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional