You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-23 - 05:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot8s.osadl.org (updated Sun Nov 23, 2025 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8020329911900,1186cyclictest9564352sleep321:10:013
81616121060,3sleep244-21ksoftirqd/219:15:212
12060822330,2sleep014-21ksoftirqd/023:42:500
802032992118,1cyclictest0-21swapper/323:15:173
802032992017,1cyclictest0-21swapper/322:35:013
80203299187,9cyclictest980163-21turbostat21:29:553
80203299187,9cyclictest980163-21turbostat21:29:553
802032991815,1cyclictest0-21swapper/320:55:233
80203299177,7cyclictest1231199-21turbostat00:00:003
802032991713,1cyclictest0-21swapper/319:40:003
80203299171,12cyclictest1101404-21turbostat22:39:593
190898170,2rtkit-daemon0-21swapper/223:46:112
80203299169,4cyclictest1279835-21turbostat00:30:003
802032991613,1cyclictest0-21swapper/323:21:503
802032991613,1cyclictest0-21swapper/322:58:073
802032991613,1cyclictest0-21swapper/300:19:173
802032991513,1cyclictest0-21swapper/323:39:403
802032991513,1cyclictest0-21swapper/322:30:013
802032991513,1cyclictest0-21swapper/320:34:003
802032991512,1cyclictest0-21swapper/323:30:303
802032991512,1cyclictest0-21swapper/321:51:283
802032991512,1cyclictest0-21swapper/320:11:533
802032991512,1cyclictest0-21swapper/300:38:383
80203299151,11cyclictest9501392sleep321:05:133
80203299151,11cyclictest874545-21turbostat20:05:013
80203299151,11cyclictest802313-21taskset19:10:003
80203299150,12cyclictest899602-21turbostat20:25:003
80203299147,4cyclictest1076988-21turbostat22:25:013
80203299145,7cyclictest849562-21taskset19:45:153
80203299141,9cyclictest8428902sleep319:40:153
802032991413,0cyclictest0-21swapper/323:41:443
802032991413,0cyclictest0-21swapper/320:28:513
802032991412,1cyclictest0-21swapper/322:14:573
802032991412,1cyclictest0-21swapper/322:08:393
802032991412,1cyclictest0-21swapper/300:32:283
802032991412,1cyclictest0-21swapper/300:08:133
802032991412,0cyclictest0-21swapper/319:56:553
80203299141,11cyclictest930733-21taskset20:46:453
802032991411,1cyclictest0-21swapper/300:11:303
80203299141,10cyclictest933344-21taskset20:50:183
80203299141,10cyclictest809320-21taskset19:15:023
80203299141,10cyclictest1264186-21taskset00:20:153
80203299141,10cyclictest1222369-21taskset23:53:113
80203299141,10cyclictest1156708-21chrt23:10:553
80203299141,10cyclictest1108177-21taskset22:41:503
80203299141,10cyclictest1035506-21taskset21:57:573
80203299141,10cyclictest1012884-21taskset21:45:163
80203299140,11cyclictest1061183-21chrt22:15:133
80203299134,6cyclictest12392022sleep300:04:463
80203299131,9cyclictest980163-21turbostat21:25:003
80203299131,9cyclictest823991-21taskset19:25:163
80203299131,9cyclictest1175027-21taskset23:25:143
80203299131,9cyclictest1036869-21taskset22:00:023
80203299131,9cyclictest1004511-21taskset21:40:033
802032991312,0cyclictest0-21swapper/319:51:543
802032991311,1cyclictest0-21swapper/323:04:013
802032991311,1cyclictest0-21swapper/321:15:153
802032991311,0cyclictest0-21swapper/321:02:423
80203299130,10cyclictest924201-21taskset20:40:243
80203299130,10cyclictest890593-21taskset20:15:193
80203299130,10cyclictest822178-21taskset19:20:193
80203299130,10cyclictest1148616-21taskset23:05:503
190898130,9rtkit-daemon1246209-21ssh00:07:190
190898130,1rtkit-daemon0-21swapper/023:09:370
8238342121,4sleep2823990-21diskstats19:25:162
80203299129,1cyclictest0-21swapper/320:05:163
80203299124,6cyclictest11242722sleep322:51:493
80203299121,9cyclictest918229-21chrt20:37:403
80203299121,9cyclictest1002380-21taskset21:35:383
80203299121,8cyclictest1207232-21taskset23:45:013
802032991211,0cyclictest0-21swapper/319:30:033
802032991210,1cyclictest0-21swapper/322:45:173
80203299120,9cyclictest988387-21taskset21:30:003
190898120,7rtkit-daemon1183313-21perl23:30:130
190898120,3rtkit-daemon667-21snmpd21:22:561
190898110,6rtkit-daemon964334-21sed21:15:002
190898110,6rtkit-daemon1280345-21df00:30:140
190898110,3rtkit-daemon808657-21munin-node19:26:200
190898110,3rtkit-daemon1084377-21ssh22:28:020
8986902101,5sleep0898748-21ntp_states20:20:230
190898100,6rtkit-daemon1180971-21ssh23:26:020
190898100,5rtkit-daemon1264177-21apt-get00:20:150
190898100,4rtkit-daemon667-21snmpd20:40:482
190898100,4rtkit-daemon667-21snmpd20:06:160
190898100,3rtkit-daemon667-21snmpd00:13:250
190898100,3rtkit-daemon1928-21sendmail-mta22:59:010
12135492100,2sleep10-21swapper/123:46:241
11261752101,7sleep21126182-21/usr/sbin/munin22:55:122
11256332101,6sleep21125689-21munin-run22:54:592
801149290,5sleep0801393-21users19:05:260
19089890,6rtkit-daemon667-21snmpd23:14:410
19089890,6rtkit-daemon319-21systemd-udevd20:29:382
19089890,5rtkit-daemon667-21snmpd21:17:491
19089890,5rtkit-daemon438-21wpa_supplicant19:12:430
19089890,5rtkit-daemon1035055-21ssh21:57:000
19089890,4rtkit-daemon1197612-21ssh23:37:030
19089890,4rtkit-daemon1164965-21ssh23:16:230
19089890,1rtkit-daemon0-21swapper/220:55:112
19089890,1rtkit-daemon0-21swapper/123:58:371
19089890,1rtkit-daemon0-21swapper/021:40:290
19089890,1rtkit-daemon0-21swapper/019:54:430
1262910294,3sleep11262916-21ssh00:18:381
1262582290,1sleep21262590-21ssh00:17:472
1223156290,6sleep21223164-21ssh23:54:592
1198100290,7sleep21198108-21ssh23:38:122
1055700290,6sleep01057781-21grep22:10:180
1026685291,6sleep21026693-21ssh21:51:052
1025103291,5sleep11025773-21cat21:50:191
1018534290,1sleep00-21swapper/021:45:460
921265280,5sleep1923404-21cut20:40:191
917381280,5sleep0917522-21sed20:35:210
911265280,4sleep1911321-21ntp_states20:30:211
887338281,4sleep2887493-21diskstats20:15:162
881049281,3sleep0881082-21apt-get20:10:130
873848281,4sleep2874074-21sed20:00:262
848151280,5sleep1848285-21sed19:40:251
842926281,4sleep0843081-21diskstats19:40:150
822666281,4sleep0822910-21users19:20:250
795801281,3sleep20-21swapper/219:05:122
19089880,5rtkit-daemon808657-21munin-node19:35:140
19089880,5rtkit-daemon1928-21sendmail-mta22:20:210
19089880,5rtkit-daemon1928-21sendmail-mta19:16:510
19089880,4rtkit-daemon912053-21chrt20:33:372
19089880,4rtkit-daemon81364-21packagekitd20:39:302
19089880,4rtkit-daemon808657-21munin-node19:47:330
19089880,4rtkit-daemon671-21ntpd22:00:180
19089880,4rtkit-daemon667-21snmpd22:18:530
19089880,4rtkit-daemon667-21snmpd00:37:200
19089880,4rtkit-daemon438-21wpa_supplicant23:52:381
19089880,4rtkit-daemon438-21wpa_supplicant21:26:101
19089880,4rtkit-daemon438-21wpa_supplicant21:26:101
19089880,4rtkit-daemon436-21systemd-logind00:18:370
19089880,4rtkit-daemon421-21avahi-daemon20:17:020
19089880,4rtkit-daemon421-21avahi-daemon19:34:000
19089880,4rtkit-daemon1108207-21ssh22:41:570
19089880,4rtkit-daemon1099912-21ssh22:36:350
19089880,3rtkit-daemon783019-21kworker/u16:0+events_power_efficient20:53:062
19089880,3rtkit-daemon436-21systemd-logind23:03:370
19089880,3rtkit-daemon1091547-21ssh22:30:580
19089880,1rtkit-daemon1907-21rtkit-daemon22:09:450
19089880,1rtkit-daemon0-21swapper/100:01:171
19089880,1rtkit-daemon0-21swapper/023:22:530
1269291281,4sleep20-21swapper/200:20:232
1247094280,1sleep20-21swapper/200:09:222
1231073280,1sleep00-21swapper/023:59:420
1157885281,5sleep11157891-21ssh23:13:431
1117363280,5sleep01117371-21ssh22:49:330
1053133281,3sleep21053170-21apt-get22:10:112
1036058280,1sleep10-21swapper/121:59:171
1017957281,4sleep21018034-21find21:45:242
999167271,4sleep21000729-21ssh21:35:222
989605270,4sleep1990795-21grep21:30:161
886207270,4sleep2886290-21/usr/sbin/munin20:10:232
795836270,4sleep30-21swapper/319:05:133
19089870,4rtkit-daemon808657-21munin-node23:31:321
19089870,4rtkit-daemon808657-21munin-node21:37:590
19089870,4rtkit-daemon808657-21munin-node21:03:292
19089870,4rtkit-daemon473-21gmain19:57:140
19089870,4rtkit-daemon473-21gmain00:01:420
19089870,3rtkit-daemon808657-21munin-node22:51:020
19089870,3rtkit-daemon808657-21munin-node20:46:002
19089870,3rtkit-daemon808657-21munin-node20:02:410
19089870,3rtkit-daemon421-21avahi-daemon21:54:040
19089870,3rtkit-daemon1928-21sendmail-mta00:26:010
19089870,3rtkit-daemon1000729-21ssh21:35:221
1246046270,4sleep11246047-21sh00:06:561
1207521270,4sleep01207642-21perl23:45:140
1196562271,4sleep11191085-21/usr/sbin/munin23:35:221
1174983271,3sleep21175024-21apt-get23:25:142
1165590270,5sleep21165598-21ssh23:17:512
1141109270,5sleep11141117-21ssh23:02:241
1141083270,1sleep20-21swapper/223:02:212
1132525270,1sleep10-21swapper/122:56:471
1103088271,3sleep20-21swapper/222:40:172
1076466271,4sleep21076468-21ssh22:23:552
1067338270,5sleep11067346-21ssh22:16:201
1061775271,3sleep21062669-21ps22:15:172
1004586271,2sleep11004623-21apt-get21:40:141
987888260,3sleep2987896-21ssh21:29:212
987888260,3sleep2987896-21ssh21:29:212
972666261,2sleep20-21swapper/221:20:152
972465261,2sleep00-21swapper/021:20:130
962452261,2sleep00-21swapper/021:10:590
956181260,3sleep1956213-21awk21:10:001
931244260,2sleep1931277-21apt-get20:50:131
887789261,2sleep10-21swapper/120:15:191
882039261,2sleep10-21swapper/120:10:181
863671261,2sleep20-21swapper/219:55:212
808258260,1sleep20-21swapper/219:13:252
19089860,3rtkit-daemon667-21snmpd21:05:162
19089860,3rtkit-daemon473-21gmain21:15:062
1295948260,1sleep20-21swapper/200:39:582
1288250260,2sleep10-21swapper/100:35:121
1286285260,1sleep10-21swapper/100:31:161
1275605260,1sleep10-21swapper/100:25:191
1220602260,3sleep01220663-21ntp_states23:50:200
1204818260,1sleep20-21swapper/223:40:242
1183213261,2sleep21183409-21ssh23:30:142
1172753260,1sleep10-21swapper/123:20:401
1172091261,2sleep20-21swapper/223:20:212
1161209260,4sleep11163260-21kthreadcore23:15:191
1156083260,1sleep20-21swapper/223:10:232
1148443260,1sleep10-21swapper/123:05:291
1145963261,3sleep21147662-21cat23:05:182
1124161260,3sleep11124167-21ssh22:51:341
1116493260,1sleep20-21swapper/222:47:232
1102212260,4sleep11103087-21sed22:40:161
1092145260,1sleep10-21swapper/122:32:231
1091427260,1sleep20-21swapper/222:30:442
1084093261,3sleep2987313-21kworker/u16:0+nfsiod22:27:192
1048730260,4sleep11050134-21memory22:05:181
1043010260,1sleep10-21swapper/122:00:571
1028695261,2sleep20-21swapper/221:55:132
1010759260,1sleep20-21swapper/221:41:232
995624250,1sleep20-21swapper/221:33:362
988896250,3sleep0989604-21irqrtprio21:30:150
962898250,1sleep10-21swapper/121:12:001
945579250,3sleep1947604-21grep21:00:191
944708250,3sleep0945578-21cut21:00:180
937542250,2sleep00-21swapper/020:55:130
937045250,1sleep00-21swapper/020:51:560
930983250,1sleep10-21swapper/120:49:561
930870250,1sleep00-21swapper/020:48:310
918232250,3sleep10-21swapper/120:37:431
906537251,2sleep00-21swapper/020:30:170
899844250,2sleep00-21swapper/020:25:160
899403250,1sleep20-21swapper/220:22:412
895071250,3sleep1896924-21kthreadcore20:20:201
880581250,1sleep10-21swapper/120:07:171
875765251,2sleep20-21swapper/220:05:182
858748250,3sleep1860613-21sed19:50:191
854946250,1sleep10-21swapper/119:47:181
848657250,1sleep20-21swapper/219:41:502
842572250,1sleep20-21swapper/219:39:342
836588250,1sleep10-21swapper/119:35:121
836124250,1sleep20-21swapper/219:32:212
818739250,3sleep1820181-21cut19:20:181
817471250,1sleep20-21swapper/219:20:152
816096250,3sleep1816160-21/usr/sbin/munin19:15:211
808382250,2sleep1808657-21munin-node19:15:011
800867251,1sleep10-21swapper/119:05:231
1287827250,3sleep20-21swapper/200:34:502
1272543250,2sleep20-21swapper/200:25:162
1270506250,3sleep11270508-21ssh00:21:551
1254463250,3sleep20-21swapper/200:12:462
1239318251,2sleep20-21swapper/200:05:012
1229726250,3sleep21229734-21ssh23:56:342
1206278250,3sleep11206280-21ssh23:43:241
1181702250,3sleep11181710-21ssh23:27:451
1117225250,1sleep10-21swapper/122:49:141
1099637250,3sleep11099645-21sh22:35:521
1083584250,1sleep10-21swapper/122:26:051
1069225250,2sleep11069260-21apt-get22:20:131
1051255250,3sleep20-21swapper/222:06:192
1044217250,3sleep21044225-21ssh22:03:472
980415240,1sleep00-21swapper/021:25:140
980415240,1sleep00-21swapper/021:25:130
971829240,3sleep00-21swapper/021:19:210
951597240,2sleep00-21swapper/021:05:180
938523240,2sleep10-21swapper/120:55:181
861553240,3sleep20-21swapper/219:53:112
849361240,2sleep2849561-21smartctl19:45:152
830374240,2sleep10-21swapper/119:30:151
829620240,3sleep10-21swapper/119:27:301
1100405240,3sleep20-21swapper/222:37:442
1059990240,2sleep11059998-21ssh22:13:001
918728230,1sleep00-21swapper/020:40:140
899880230,1sleep10-21swapper/120:25:171
874246230,2sleep10-21swapper/120:01:251
862249230,1sleep10-21swapper/119:55:161
1247687230,1sleep10-21swapper/100:10:121
1012584230,1sleep10-21swapper/121:45:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional