You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-20 - 11:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot8s.osadl.org (updated Sat Dec 20, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6478919912910,1286cyclictest875534-21sleep22:05:003
647891999101,904cyclictest843419-21sleep21:40:003
647891994050,401cyclictest772446-21turbostat20:49:553
647891992941,290cyclictest740372-21turbostat20:24:553
647891992800,266cyclictest804586-21turbostat21:14:563
647891992623,1cyclictest0-21swapper/319:15:003
8567682210,3sleep1321rcuc/121:50:151
647891991916,1cyclictest0-21swapper/323:55:113
64789199181,12cyclictest824144-21taskset21:25:113
301798180,14rtkit-daemon654387-21munin-node20:46:020
64789199176,9cyclictest1023494-21turbostat00:04:543
647891991714,1cyclictest0-21swapper/319:55:253
64789199171,11cyclictest920725-21taskset22:40:123
64789199161,11cyclictest1068740-21taskset00:35:123
9157292152,6sleep0836352-21kworker/u16:0-kvfree_rcu_reclaim22:35:180
64789199155,8cyclictest978237-21turbostat23:29:563
64789199154,7cyclictest10044672sleep323:45:133
647891991513,0cyclictest0-21swapper/322:32:523
647891991512,1cyclictest0-21swapper/323:02:223
647891991512,1cyclictest0-21swapper/320:50:223
647891991512,1cyclictest0-21swapper/319:25:203
64789199151,11cyclictest817868-21chrt21:20:123
64789199151,11cyclictest798288-21taskset21:05:013
7632032140,6sleep1759808-21/usr/sbin/munin20:35:181
647891991413,0cyclictest0-21swapper/323:06:113
647891991413,0cyclictest0-21swapper/320:08:033
647891991413,0cyclictest0-21swapper/320:08:023
647891991412,1cyclictest0-21swapper/322:53:003
647891991412,1cyclictest0-21swapper/322:36:153
647891991412,1cyclictest0-21swapper/321:53:263
647891991412,1cyclictest0-21swapper/321:32:273
647891991412,0cyclictest0-21swapper/323:20:323
647891991412,0cyclictest0-21swapper/319:44:473
647891991411,1cyclictest0-21swapper/321:08:293
647891991411,1cyclictest0-21swapper/320:01:323
64789199141,10cyclictest984902-21taskset23:30:013
64789199141,10cyclictest752135-21taskset20:25:193
64789199141,10cyclictest734030-21taskset20:15:013
64789199141,10cyclictest1068442-21taskset00:34:563
64789199140,6cyclictest0-21swapper/319:35:233
64789199140,11cyclictest962813-21taskset23:10:183
64789199140,11cyclictest1049466-21chrt00:20:143
9019092131,5sleep0902761-21irqrtprio22:25:170
9019092131,5sleep0902761-21irqrtprio22:25:170
7534292130,3sleep0753462-21apt-get20:30:130
64789199139,2cyclictest881916-21turbostat22:09:593
64789199136,4cyclictest1061763-21taskset00:26:593
64789199134,7cyclictest933247-21taskset22:49:153
64789199131,9cyclictest998030-21taskset23:40:113
64789199131,9cyclictest888605-21chrt22:15:133
64789199131,9cyclictest888266-21taskset22:14:253
64789199131,9cyclictest739995-21taskset20:15:373
64789199131,9cyclictest1043183-21chrt00:15:133
64789199131,9cyclictest1036625-21taskset00:10:123
647891991311,1cyclictest0-21swapper/321:19:343
647891991311,1cyclictest0-21swapper/319:22:033
647891991311,1cyclictest0-21swapper/300:07:413
64789199131,10cyclictest945439-21taskset22:55:243
64789199131,10cyclictest862898-21taskset21:55:013
647891991311,0cyclictest0-21swapper/320:31:193
647891991310,1cyclictest0-21swapper/323:35:233
64789199130,6cyclictest0-21swapper/322:20:153
64789199130,10cyclictest1016309-21chrt23:50:213
64789199124,6cyclictest971604-21taskset23:17:393
64789199123,7cyclictest8500642sleep321:45:133
64789199121,9cyclictest907274-21taskset22:25:313
64789199121,9cyclictest907274-21taskset22:25:313
64789199121,9cyclictest843636-21taskset21:40:033
64789199121,9cyclictest772176-21chrt20:41:493
64789199121,9cyclictest701598-21taskset19:46:363
64789199121,9cyclictest681541-21taskset19:30:203
647891991211,0cyclictest0-21swapper/320:56:023
647891991211,0cyclictest0-21swapper/319:52:113
647891991210,1cyclictest0-21swapper/319:15:163
64789199120,9cyclictest759824-21taskset20:35:123
301798120,7rtkit-daemon1049620-21df_inode00:20:151
301798120,2rtkit-daemon695884-21diskstats19:45:131
9453132111,7sleep2939999-21/usr/sbin/munin22:55:232
301798110,6rtkit-daemon1036768-21df00:10:131
301798110,4rtkit-daemon654387-21munin-node19:42:091
301798110,4rtkit-daemon3200908-21packagekitd21:21:490
9849182102,4sleep1984951-21apt-get23:30:121
7728582100,8sleep2772891-21fschecks_count20:45:142
301798100,6rtkit-daemon413-21rpcbind19:15:151
301798100,5rtkit-daemon914465-21df_inode22:35:141
301798100,3rtkit-daemon3200908-21packagekitd20:38:180
903948291,5sleep2906083-21grep22:25:192
903948291,5sleep2906083-21grep22:25:192
864365290,6sleep1865618-21cut21:55:161
830722291,5sleep0830923-21hddtemp_smartct21:30:150
805802291,5sleep10-21swapper/121:10:171
798147291,5sleep2798287-21latency_hist21:05:002
739643290,6sleep2739763-21ls20:15:252
708701290,1sleep00-21swapper/019:55:160
641845290,4sleep30-21swapper/319:05:143
30179890,5rtkit-daemon636-21snmpd23:08:261
30179890,5rtkit-daemon636-21snmpd20:27:500
30179890,5rtkit-daemon608647-21munin-node19:12:251
30179890,5rtkit-daemon34-21ksoftirqd/119:51:101
30179890,5rtkit-daemon3205-21sendmail-mta23:23:151
30179890,5rtkit-daemon3205-21sendmail-mta20:52:350
30179890,4rtkit-daemon891723-21cut22:15:181
30179890,4rtkit-daemon647725-21taskset19:09:441
30179890,4rtkit-daemon1074116-21ps00:35:211
1012491291,5sleep21014536-21tr23:50:172
998032281,3sleep0998065-21apt-get23:40:130
992073281,4sleep0993069-21irqstats23:35:160
965046280,5sleep2965047-21sshd23:11:102
933788280,1sleep10-21swapper/122:50:141
887416280,4sleep0887475-21ntp_states22:10:220
882408280,5sleep2882597-21/usr/sbin/munin22:10:162
869385281,4sleep0869537-21df_inode22:00:120
856685280,5sleep0856767-21fschecks_time21:50:150
842520281,5sleep2842564-21proc_pri21:35:222
831505280,5sleep2832434-21tr21:30:172
722396281,5sleep1723614-21grep20:05:181
722396281,5sleep1723614-21grep20:05:181
713715280,4sleep2713792-21ntp_states19:55:222
668653281,4sleep0668859-21users19:20:210
30179880,5rtkit-daemon636-21snmpd23:35:541
30179880,5rtkit-daemon492-21gmain23:41:051
30179880,5rtkit-daemon3205-21sendmail-mta23:26:151
30179880,5rtkit-daemon0-21swapper/123:45:551
30179880,4rtkit-daemon667-21lldpd19:29:571
30179880,4rtkit-daemon654387-21munin-node21:42:000
30179880,4rtkit-daemon654387-21munin-node21:35:360
30179880,4rtkit-daemon654387-21munin-node21:07:020
30179880,4rtkit-daemon654387-21munin-node20:55:060
30179880,4rtkit-daemon654387-21munin-node19:21:061
30179880,4rtkit-daemon654387-21munin-node00:25:281
30179880,4rtkit-daemon638-21ntpd20:16:360
30179880,4rtkit-daemon636-21snmpd23:56:171
30179880,4rtkit-daemon636-21snmpd23:18:581
30179880,4rtkit-daemon636-21snmpd23:11:511
30179880,4rtkit-daemon636-21snmpd22:34:491
30179880,4rtkit-daemon636-21snmpd21:02:150
30179880,4rtkit-daemon636-21snmpd20:10:150
30179880,4rtkit-daemon636-21snmpd19:31:501
30179880,4rtkit-daemon636-21snmpd00:05:561
30179880,4rtkit-daemon492-21gmain21:18:240
30179880,4rtkit-daemon430-21avahi-daemon21:10:230
30179880,4rtkit-daemon3205-21sendmail-mta20:08:000
30179880,4rtkit-daemon3205-21sendmail-mta20:08:000
30179880,4rtkit-daemon3200908-21packagekitd00:01:091
30179880,4rtkit-daemon0-21swapper/122:11:051
30179880,3rtkit-daemon740655-21apt-get20:20:130
30179880,1rtkit-daemon0-21swapper/122:09:091
30179880,1rtkit-daemon0-21swapper/122:03:361
30179880,1rtkit-daemon0-21swapper/119:55:121
30179880,1rtkit-daemon0-21swapper/100:34:141
30179880,1rtkit-daemon0-21swapper/021:47:580
30179880,1rtkit-daemon0-21swapper/020:02:350
1043442280,6sleep11044354-21cut00:15:151
1017326281,4sleep01017480-21df_inode23:55:120
991513271,2sleep20-21swapper/223:35:012
972463271,3sleep2972910-21cut23:20:152
970863270,4sleep2970940-21ntp_states23:15:232
954673272,3sleep2956752-21tr23:05:172
940996271,3sleep1942096-21tr22:55:181
895440270,5sleep2896172-21sed22:20:132
795761270,4sleep1797113-21memory21:00:181
771607270,3sleep1771732-21python320:40:201
741167270,4sleep2742136-21grep20:20:172
720231271,3sleep2714917-21/usr/sbin/munin20:00:222
687968270,5sleep2688115-21tr19:35:222
683018271,3sleep1683219-21hddtemp_smartct19:35:171
648454270,4sleep2649042-21ps19:10:142
30179870,4rtkit-daemon654387-21munin-node21:55:560
30179870,4rtkit-daemon636-21snmpd22:45:351
30179870,4rtkit-daemon492-21gmain22:20:521
30179870,3rtkit-daemon654387-21munin-node21:27:520
30179870,3rtkit-daemon636-21snmpd23:50:211
30179870,3rtkit-daemon636-21snmpd20:42:050
30179870,3rtkit-daemon492-21gmain22:40:331
30179870,2rtkit-daemon952176-21users23:00:251
30179870,1rtkit-daemon0-21swapper/122:25:351
30179870,1rtkit-daemon0-21swapper/122:25:351
1026598270,4sleep01028788-21grep00:00:200
979937261,2sleep00-21swapper/023:25:190
971513260,1sleep00-21swapper/023:16:330
964735260,3sleep0964979-21users23:10:230
953197260,4sleep0953709-21irqcore23:05:150
952521260,1sleep20-21swapper/223:04:452
947902260,3sleep0949831-21cut23:00:180
939998260,2sleep0940031-21apt-get22:55:140
936076260,3sleep0938254-21grep22:50:170
933305260,3sleep0933325-21grep22:50:010
913205260,4sleep2913330-21python322:30:232
900394260,3sleep0900519-21python322:20:200
888763260,3sleep0888957-21grep22:15:150
868254260,3sleep2868379-21python321:55:222
853366260,3sleep1855177-21grep21:45:191
836000260,1sleep10-21swapper/121:30:241
824108261,2sleep20-21swapper/221:25:112
817728260,4sleep2817866-21expr21:20:122
811320261,3sleep2811236-21/usr/sbin/munin21:15:152
804560261,3sleep10-21swapper/121:10:011
779567261,2sleep20-21swapper/220:50:152
747210260,3sleep2747409-21hddtemp_smartct20:25:152
741015261,2sleep1741164-21irqcore20:20:161
740342260,4sleep1740406-21systemd-run20:20:011
695196260,1sleep00-21swapper/019:41:570
677633262,2sleep2679482-21kthreadcore19:30:172
1051947261,2sleep00-21swapper/000:20:200
1049430261,2sleep20-21swapper/200:20:142
1044356260,3sleep21045521-21kthreadcore00:15:162
1042701260,1sleep00-21swapper/000:14:270
1034126260,4sleep21035297-21memory00:05:192
1010543260,1sleep00-21swapper/023:48:590
986372250,3sleep2987785-21kthreadcore23:30:172
973879251,2sleep00-21swapper/023:20:180
933558250,2sleep20-21swapper/222:50:112
923834250,3sleep0925762-21cat22:40:190
920689250,2sleep20-21swapper/222:40:122
907888250,2sleep00-21swapper/022:30:120
894452250,1sleep20-21swapper/222:16:132
856499250,2sleep20-21swapper/221:50:132
850410251,2sleep20-21swapper/221:45:162
849716250,1sleep20-21swapper/221:43:512
849490250,1sleep10-21swapper/121:40:571
843289250,1sleep10-21swapper/121:38:581
817692250,2sleep10-21swapper/121:20:111
817209250,1sleep10-21swapper/121:17:171
806816250,3sleep2808906-21cut21:10:182
798415250,2sleep20-21swapper/221:05:132
791365250,1sleep10-21swapper/120:55:351
787391250,2sleep20-21swapper/220:55:202
785257250,1sleep10-21swapper/120:54:301
778607250,1sleep10-21swapper/120:46:451
747095250,2sleep10-21swapper/120:25:141
733784250,1sleep10-21swapper/120:13:321
728146251,2sleep20-21swapper/220:10:162
723616250,3sleep2725735-21tr20:05:192
723616250,3sleep2725735-21tr20:05:192
720874250,1sleep10-21swapper/120:02:451
708238250,1sleep00-21swapper/019:54:510
701534250,1sleep00-21swapper/019:45:500
701168250,3sleep2695692-21/usr/sbin/munin19:45:232
689556250,2sleep2689731-21if_eno119:40:152
688755250,1sleep00-21swapper/019:36:520
675353250,3sleep00-21swapper/019:26:350
669486250,2sleep20-21swapper/219:25:122
664822251,2sleep20-21swapper/219:20:162
658122251,2sleep20-21swapper/219:15:192
643632250,2sleep20-21swapper/219:05:162
1068704250,2sleep20-21swapper/200:35:122
1048797250,1sleep00-21swapper/000:15:310
1037030250,2sleep20-21swapper/200:10:162
1031093250,2sleep00-21swapper/000:05:170
1029838250,1sleep20-21swapper/200:03:582
1023235250,1sleep20-21swapper/223:56:582
1011274250,2sleep00-21swapper/023:50:140
1000228250,2sleep20-21swapper/223:40:192
990374240,2sleep00-21swapper/023:30:230
933090240,3sleep20-21swapper/222:47:142
881587240,3sleep20-21swapper/222:06:022
830063240,1sleep10-21swapper/121:27:241
765836240,3sleep20-21swapper/220:38:052
759315240,3sleep20-21swapper/220:32:002
754886240,1sleep10-21swapper/120:30:181
654238240,2sleep0654728-21kthreadcore19:15:010
654155240,3sleep00-21swapper/019:13:570
641644240,1sleep00-21swapper/019:05:110
1074632240,3sleep00-21swapper/000:36:460
1066455241,1sleep00-21swapper/000:30:200
1058611240,2sleep21060806-21kthreadcore00:25:192
1055843240,1sleep00-21swapper/000:25:130
1010363240,1sleep20-21swapper/223:46:402
983559230,1sleep20-21swapper/223:25:212
914554230,1sleep20-21swapper/222:35:162
875953230,1sleep00-21swapper/022:05:130
869345230,1sleep20-21swapper/222:00:102
771159230,2sleep20-21swapper/220:40:172
702153230,1sleep20-21swapper/219:50:132
676140230,1sleep00-21swapper/019:30:110
1063253230,1sleep20-21swapper/200:30:192
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional