You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-08 - 19:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sat Nov 08, 2025 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2785599250,1cyclictest17172-21sh07:54:592
2785999240,23cyclictest0-21swapper/310:25:583
27855992422,1cyclictest3756-21apt-get10:45:572
27851992423,1cyclictest10327-21apt-get07:40:561
27851992421,2cyclictest22656-21munin-plugin-st11:25:001
2785999230,2cyclictest3742-21turbostat.cron09:40:003
2785999230,22cyclictest6864-21perf08:40:003
2785999230,22cyclictest15435-21perf11:10:003
2785599231,21cyclictest6884-21basename08:40:002
2785599230,0cyclictest0-21swapper/209:05:592
2785199230,23cyclictest0-21swapper/111:25:021
2785999221,20cyclictest0-21swapper/310:55:013
2785999220,21cyclictest562-21apt-get07:20:563
2785999220,21cyclictest10970-21apt-get11:00:583
2785999220,21cyclictest0-21swapper/310:35:573
2785999220,1cyclictest21169-21apt-get10:15:013
27855992220,1cyclictest576-21apt-get10:40:002
2785599220,21cyclictest0-21swapper/208:20:002
2785599220,1cyclictest576-21apt-get10:40:022
2785599220,1cyclictest576-21apt-get10:40:012
2785599220,1cyclictest5042-21cron07:30:002
27851992222,0cyclictest590-21apt-get11:45:591
27851992219,2cyclictest20540-21sh10:15:001
27851992219,2cyclictest10946-21apt09:55:001
2785199220,21cyclictest0-21swapper/111:15:591
2785199220,21cyclictest0-21swapper/110:50:591
2785199220,21cyclictest0-21swapper/107:50:571
2785199220,21cyclictest0-21swapper/107:10:591
2785199220,1cyclictest3033-21apt-get11:51:001
2785199220,1cyclictest23670-21apt-get09:15:021
2785199220,1cyclictest17162-21idleruntime-cro12:20:001
27847992222,0cyclictest0-21swapper/007:40:570
27847992221,1cyclictest590-21perf10:40:000
27847992221,1cyclictest5433-21apt-get11:56:020
27847992221,0cyclictest0-21swapper/009:31:000
27847992220,1cyclictest26486-21perf09:20:000
27847992220,1cyclictest24367-21latency_hist12:35:000
27847992219,2cyclictest20533-21apt-get10:15:000
27847992219,2cyclictest19576-21apt12:25:000
2784799220,21cyclictest596-21in:imuxsock11:40:010
2784799220,21cyclictest28963-21idleruntime-cro09:24:590
2784799220,21cyclictest20229-21sendmail-msp11:19:590
2784799220,21cyclictest0-21swapper/009:15:000
27859992121,0cyclictest0-21swapper/310:06:003
27859992121,0cyclictest0-21swapper/309:15:593
27859992120,1cyclictest30003-21apt-get08:20:593
27859992119,2cyclictest5406-21apt-get07:30:583
27859992119,1cyclictest17518-21apt-get07:55:583
2785999210,20cyclictest15760-21apt-get11:10:583
2785999210,20cyclictest0-21swapper/309:20:583
2785999210,20cyclictest0-21swapper/309:05:583
2785999210,20cyclictest0-21swapper/308:55:583
2785999210,0cyclictest0-21swapper/309:55:583
2785999210,0cyclictest0-21swapper/308:30:583
27855992121,0cyclictest18157-21apt-get11:15:592
27855992119,1cyclictest593-21dbus-daemon10:25:002
27855992119,1cyclictest32065-21apt08:25:012
2785599210,21cyclictest0-21swapper/212:05:592
2785599210,21cyclictest0-21swapper/207:30:582
2785599210,20cyclictest0-21swapper/212:15:582
2785599210,20cyclictest0-21swapper/211:10:582
2785599210,20cyclictest0-21swapper/209:35:582
2785599210,0cyclictest6153-21apt-get10:50:582
2785599210,0cyclictest12719-21apt-get07:45:562
2785599210,0cyclictest12690-21apt-get12:10:582
2785599210,0cyclictest0-21swapper/211:35:582
2785599210,0cyclictest0-21swapper/211:05:592
27851992121,0cyclictest30597-21apt-get07:15:561
27851992120,0cyclictest0-21swapper/111:56:021
2785199210,21cyclictest0-21swapper/109:55:581
2785199210,20cyclictest0-21swapper/112:00:571
2785199210,1cyclictest19904-21apt-get12:26:001
2785199210,0cyclictest0-21swapper/112:30:591
27847992121,0cyclictest13695-21apt-get10:00:580
2784799210,20cyclictest0-21swapper/010:25:570
2784799210,1cyclictest4459-21perf08:35:000
2784799210,1cyclictest0-21swapper/011:45:590
2784799210,0cyclictest0-21swapper/012:05:590
2784799210,0cyclictest0-21swapper/009:25:590
27859992020,0cyclictest9600-21apt-get08:45:583
27859992020,0cyclictest32401-21apt-get08:26:013
27859992020,0cyclictest0-21swapper/312:20:593
27859992020,0cyclictest0-21swapper/311:40:593
27859992020,0cyclictest0-21swapper/311:25:023
27859992020,0cyclictest0-21swapper/310:00:573
27859992020,0cyclictest0-21swapper/308:41:003
27859992019,1cyclictest0-21swapper/307:40:573
27859992018,1cyclictest0-21swapper/312:26:003
27859992017,1cyclictest313-21systemd-journal07:19:593
2785999200,20cyclictest0-21swapper/312:15:583
2785999200,20cyclictest0-21swapper/308:05:573
2785999200,20cyclictest0-21swapper/307:10:593
2785999200,19cyclictest30609-21cron10:35:003
2785999200,19cyclictest0-21swapper/311:50:593
2785999200,0cyclictest0-21swapper/312:10:583
27855992020,0cyclictest0-21swapper/211:56:012
27855992020,0cyclictest0-21swapper/211:45:582
27855992020,0cyclictest0-21swapper/211:20:582
27855992020,0cyclictest0-21swapper/210:10:592
27855992020,0cyclictest0-21swapper/208:26:012
27855992020,0cyclictest0-21swapper/207:40:572
27855992019,1cyclictest0-21swapper/209:00:002
27855992019,0cyclictest0-21swapper/208:10:582
27855992018,1cyclictest20533-21apt-get10:15:012
2785599200,20cyclictest0-21swapper/209:15:582
2785599200,20cyclictest0-21swapper/208:50:582
2785599200,20cyclictest0-21swapper/207:55:582
2785599200,0cyclictest0-21swapper/210:56:002
27851992020,0cyclictest19208-21apt-get09:05:591
27851992020,0cyclictest10970-21apt-get11:00:591
27851992020,0cyclictest10295-21apt-get12:05:591
27851992020,0cyclictest0-21swapper/110:40:581
27851992020,0cyclictest0-21swapper/110:40:571
27851992020,0cyclictest0-21swapper/110:15:581
27851992020,0cyclictest0-21swapper/109:45:571
27851992020,0cyclictest0-21swapper/108:45:581
27851992020,0cyclictest0-21swapper/108:26:011
27851992020,0cyclictest0-21swapper/108:20:031
27851992018,1cyclictest30618-21apt10:34:591
2785199200,0cyclictest0-21swapper/110:00:581
27847992020,0cyclictest6480-21apt-get09:45:580
27847992020,0cyclictest29271-21apt-get08:20:020
27847992020,0cyclictest0-21swapper/011:10:580
27847992020,0cyclictest0-21swapper/007:35:560
27847992020,0cyclictest0-21swapper/007:15:560
27847992018,1cyclictest26799-21munin-run12:40:000
2784799200,19cyclictest8878-21apt-get09:50:580
2784799200,0cyclictest0-21swapper/010:50:590
2784799200,0cyclictest0-21swapper/010:20:580
2784799200,0cyclictest0-21swapper/008:50:580
27859991919,0cyclictest0-21swapper/311:20:583
27859991919,0cyclictest0-21swapper/310:40:583
27859991919,0cyclictest0-21swapper/310:40:583
27859991919,0cyclictest0-21swapper/310:20:593
27859991919,0cyclictest0-21swapper/309:45:583
2785999190,1cyclictest0-21swapper/309:35:003
2785999190,0cyclictest0-21swapper/310:55:593
27855991919,0cyclictest26822-21apt-get09:20:582
27855991919,0cyclictest0-21swapper/212:26:002
27855991919,0cyclictest0-21swapper/210:30:582
27855991919,0cyclictest0-21swapper/209:25:592
27855991919,0cyclictest0-21swapper/207:10:582
27855991917,1cyclictest25447-21perf11:30:002
2785599190,18cyclictest32705-21dump-pmu-power11:45:012
27851991919,0cyclictest0-21swapper/109:20:581
27851991918,1cyclictest29281-21apt-get09:25:591
27851991918,1cyclictest0-21swapper/107:25:591
2785199190,18cyclictest26794-21sendmail-msp12:40:011
27847991919,0cyclictest16094-21apt-get10:06:010
27847991919,0cyclictest0-21swapper/011:00:590
27847991919,0cyclictest0-21swapper/009:35:580
27847991919,0cyclictest0-21swapper/008:55:590
27847991919,0cyclictest0-21swapper/008:25:590
2784799190,1cyclictest0-21swapper/008:05:000
2784799190,18cyclictest0-21swapper/007:50:000
27859991818,0cyclictest16812-21apt-get09:00:583
27859991818,0cyclictest0-21swapper/312:30:593
27859991818,0cyclictest0-21swapper/311:31:013
27859991818,0cyclictest0-21swapper/311:31:013
27859991818,0cyclictest0-21swapper/310:10:583
27859991818,0cyclictest0-21swapper/309:10:583
27859991818,0cyclictest0-21swapper/308:15:593
27859991817,0cyclictest10295-21apt-get12:05:583
27855991818,0cyclictest0-21swapper/210:06:012
27855991818,0cyclictest0-21swapper/207:15:562
27855991817,1cyclictest2387-21apt-get08:30:582
27855991817,0cyclictest31680-21apt-get09:31:002
27851991818,0cyclictest0-21swapper/112:20:591
27851991818,0cyclictest0-21swapper/111:40:581
27851991818,0cyclictest0-21swapper/111:05:591
27851991818,0cyclictest0-21swapper/110:56:001
27851991818,0cyclictest0-21swapper/109:00:591
27851991818,0cyclictest0-21swapper/108:05:581
27851991818,0cyclictest0-21swapper/107:20:571
27851991817,0cyclictest19939-21apt-get08:01:011
2785199180,17cyclictest28173-21apt-get11:35:571
2785199180,17cyclictest0-21swapper/110:25:001
27847991818,0cyclictest30647-21apt-get11:40:590
27847991818,0cyclictest22329-21apt-get08:05:580
27847991818,0cyclictest0-21swapper/012:25:590
27847991818,0cyclictest0-21swapper/011:51:000
27847991818,0cyclictest0-21swapper/011:20:570
27847991818,0cyclictest0-21swapper/009:40:580
27847991818,0cyclictest0-21swapper/007:55:580
27847991818,0cyclictest0-21swapper/007:30:570
27847991817,1cyclictest28502-21apt-get10:30:580
27847991817,1cyclictest0-21swapper/012:00:580
27847991817,0cyclictest15086-21apt-get12:15:580
27847991817,0cyclictest0-21swapper/011:26:000
2784799180,1cyclictest11276-21apt-get09:55:580
2784799180,18cyclictest0-21swapper/010:47:490
27859991717,0cyclictest0-21swapper/311:20:003
27859991717,0cyclictest0-21swapper/309:26:003
27859991716,0cyclictest0-21swapper/311:35:583
2785999170,17cyclictest0-21swapper/307:50:573
27855991717,0cyclictest8878-21apt-get09:50:562
27855991717,0cyclictest7897-21apt-get12:00:572
27855991717,0cyclictest0-21swapper/212:35:592
27855991717,0cyclictest0-21swapper/212:20:592
27855991717,0cyclictest0-21swapper/207:35:572
27851991717,0cyclictest0-21swapper/111:30:591
27851991717,0cyclictest0-21swapper/111:30:591
27851991717,0cyclictest0-21swapper/110:45:581
27851991717,0cyclictest0-21swapper/107:45:541
27851991716,1cyclictest0-21swapper/108:30:571
27847991717,0cyclictest1347-21apt-get10:40:580
27847991717,0cyclictest1347-21apt-get10:40:580
27847991717,0cyclictest0-21swapper/012:10:590
27847991715,1cyclictest21268-21latency_hist09:10:000
27859991616,0cyclictest0-21swapper/307:45:563
27859991615,1cyclictest331rcuc/308:01:003
27859991614,1cyclictest5826-21perf10:50:013
2785599160,15cyclictest8380-21awk08:41:062
2785599160,15cyclictest24102-21sshd12:31:592
27851991616,0cyclictest0-21swapper/111:10:581
27851991616,0cyclictest0-21swapper/109:40:571
27851991616,0cyclictest0-21swapper/108:50:581
27851991615,0cyclictest0-21swapper/112:10:591
27851991615,0cyclictest0-21swapper/110:25:581
2785199160,2cyclictest8100-21grep07:35:571
2785199160,15cyclictest6864-21perf08:39:591
27847991614,1cyclictest8552-21apt-get10:56:000
2784799160,8cyclictest3007-21apt-get07:25:580
2784799160,15cyclictest1398-21irqstats07:21:000
2784799160,15cyclictest0-21swapper/010:16:020
27859991515,0cyclictest0-21swapper/312:00:583
27859991514,0cyclictest0-21swapper/312:35:593
27859991513,1cyclictest11995-21apt-get08:50:573
27855991515,0cyclictest0-21swapper/211:31:012
27855991515,0cyclictest0-21swapper/211:31:012
27855991515,0cyclictest0-21swapper/207:20:562
27855991514,0cyclictest26095-21apt-get10:25:572
2785599150,14cyclictest0-21swapper/211:00:592
2785599150,14cyclictest0-21swapper/209:59:062
27851991515,0cyclictest0-21swapper/110:06:011
27851991515,0cyclictest0-21swapper/107:30:581
27851991514,0cyclictest0-21swapper/108:15:581
27847991513,1cyclictest27491-21users11:31:100
27847991513,1cyclictest27491-21users11:31:100
27847991513,1cyclictest24727-21apt-get08:10:580
27859991414,0cyclictest0-21swapper/311:56:013
27859991414,0cyclictest0-21swapper/309:50:573
27859991414,0cyclictest0-21swapper/307:35:513
2785999140,6cyclictest0-21swapper/308:10:583
2785999140,14cyclictest0-21swapper/311:46:053
2785999140,14cyclictest0-21swapper/309:40:343
2785999140,14cyclictest0-21swapper/307:28:543
2785599140,14cyclictest0-21swapper/209:45:512
2785599140,14cyclictest0-21swapper/209:40:392
2785599140,14cyclictest0-21swapper/209:03:472
2785599140,14cyclictest0-21swapper/208:45:222
2785599140,14cyclictest0-21swapper/208:05:192
2785599140,0cyclictest0-21swapper/210:02:012
2785599140,0cyclictest0-21swapper/209:11:022
27851991414,0cyclictest0-21swapper/109:30:591
27851991414,0cyclictest0-21swapper/107:55:291
27851991413,1cyclictest1662-21apt-get09:35:561
2785199140,14cyclictest0-21swapper/109:11:561
2785199140,14cyclictest0-21swapper/108:40:031
27847991414,0cyclictest13366-21apt-get11:05:590
27847991414,0cyclictest0-21swapper/009:00:590
27847991414,0cyclictest0-21swapper/008:45:590
27847991414,0cyclictest0-21swapper/008:36:560
2784799140,14cyclictest0-21swapper/008:18:170
2784799140,14cyclictest0-21swapper/007:50:050
2784799140,14cyclictest0-21swapper/007:11:410
2784799140,0cyclictest0-21swapper/008:40:260
27855991313,0cyclictest0-21swapper/208:00:492
2785199135,7cyclictest25458-21if_enp1s0.808:11:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional