You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-18 - 22:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot8s.osadl.org (updated Tue Nov 18, 2025 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6631829913541,1350cyclictest9807132sleep310:49:563
6631829912761,1270cyclictest6848052sleep307:25:013
6631829912160,1213cyclictest7805302sleep308:40:013
6631829911930,1190cyclictest7487032sleep308:15:023
6631829911720,1169cyclictest8125602sleep309:05:013
6631829911340,1129cyclictest867597-21sleep09:40:003
6631829910980,1094cyclictest7168942sleep307:50:013
66339423430,341sleep196947-21kworker/u16:1+efi_rts_wq07:10:181
108830021950,1sleep01088297-21irqstats11:55:170
663182992118,1cyclictest0-21swapper/311:50:173
663182991916,1cyclictest0-21swapper/307:15:203
663182991815,1cyclictest0-21swapper/310:05:003
663182991814,2cyclictest1135653-21turbostat12:25:003
66318299176,7cyclictest9884282sleep310:54:023
66318299176,7cyclictest8569342sleep309:30:243
663182991613,1cyclictest0-21swapper/310:25:233
663182991613,1cyclictest0-21swapper/310:18:253
663182991613,1cyclictest0-21swapper/308:05:223
663182991613,1cyclictest0-21swapper/307:35:363
66318299161,11cyclictest1136008-21taskset12:25:133
66318299155,7cyclictest954789-21taskset10:31:093
66318299154,8cyclictest735738-21turbostat08:05:013
66318299151,9cyclictest7677962sleep308:30:123
663182991512,1cyclictest0-21swapper/309:45:203
663182991512,1cyclictest0-21swapper/308:35:183
663182991512,1cyclictest0-21swapper/308:35:173
663182991512,1cyclictest0-21swapper/308:13:533
66318299151,11cyclictest964558-21turbostat10:40:013
190898150,3rtkit-daemon1105794-21kthreadcore12:05:192
190898150,2rtkit-daemon0-21swapper/108:04:431
8262322140,4sleep0826240-21ssh09:13:370
7618582141,4sleep2762590-21sed08:25:192
66318299144,7cyclictest1005498-21taskset11:05:033
663182991412,1cyclictest0-21swapper/312:31:203
663182991412,1cyclictest0-21swapper/311:33:463
663182991412,1cyclictest0-21swapper/310:40:003
663182991412,1cyclictest0-21swapper/309:10:023
663182991412,1cyclictest0-21swapper/307:46:283
663182991412,0cyclictest0-21swapper/312:01:173
663182991412,0cyclictest0-21swapper/311:18:073
663182991412,0cyclictest0-21swapper/309:40:283
66318299141,11cyclictest1061883-21taskset11:39:573
663182991411,1cyclictest0-21swapper/308:55:223
663182991411,1cyclictest0-21swapper/308:20:163
66318299141,10cyclictest890436-21chrt09:51:543
66318299141,10cyclictest761482-21taskset08:25:153
66318299141,10cyclictest696531-21taskset07:30:223
66318299141,10cyclictest1111594-21taskset12:10:143
66318299140,5cyclictest0-21swapper/307:40:013
66318299135,5cyclictest10621732sleep311:40:023
66318299134,6cyclictest10195282sleep311:10:373
66318299133,7cyclictest0-21swapper/312:15:163
66318299131,9cyclictest914949-21taskset10:07:243
66318299131,9cyclictest849858-21taskset09:26:333
66318299131,9cyclictest723467-21chrt07:55:143
66318299131,9cyclictest663346-21taskset07:10:063
66318299131,9cyclictest11033032sleep312:05:143
66318299131,9cyclictest10868152sleep311:55:143
66318299131,9cyclictest1071349-21taskset11:45:183
66318299131,9cyclictest1036518-21taskset11:22:183
66318299131,10cyclictest922295-21chrt10:10:413
66318299131,10cyclictest833434-21taskset09:16:243
66318299131,10cyclictest798995-21taskset08:50:443
66318299130,10cyclictest989074-21taskset10:55:013
66318299130,10cyclictest932455-21taskset10:20:033
66318299130,10cyclictest898157-21taskset09:56:033
10359082130,4sleep11035910-21ssh11:20:521
9458332121,8sleep0945895-21ntp_states10:25:230
66318299124,6cyclictest841371-21taskset09:20:443
66318299121,9cyclictest806191-21taskset09:00:043
66318299121,9cyclictest1043848-21taskset11:25:343
66318299121,8cyclictest997144-21chrt11:00:023
66318299121,8cyclictest786869-21taskset08:45:023
66318299121,10cyclictest1158058-21taskset12:35:443
663182991210,1cyclictest0-21swapper/307:21:473
190898120,3rtkit-daemon299-21systemd-journal10:44:421
9807392112,5sleep2980746-21cron10:50:002
9017042110,8sleep2903321-21tr10:00:192
8859272110,8sleep2887464-21ssh09:50:172
6696862111,7sleep1671395-21memory07:15:021
190898110,8rtkit-daemon473-21gmain07:38:341
190898110,4rtkit-daemon866122-21ssh09:36:380
190898110,4rtkit-daemon1117726-21ssh12:11:412
190898110,3rtkit-daemon1928-21sendmail-mta11:04:380
190898110,1rtkit-daemon0-21swapper/209:12:562
10455922111,7sleep11045600-21ssh11:29:381
9933822100,7sleep0994221-21ls10:55:190
8355392100,3sleep1835660-21perl09:20:151
7107352101,2sleep00-21swapper/007:45:140
6964062101,5sleep2696530-21python307:30:222
190898100,7rtkit-daemon421-21avahi-daemon12:39:032
190898100,6rtkit-daemon937794-21/usr/sbin/munin10:20:210
190898100,6rtkit-daemon1021012-21ssh11:13:581
190898100,5rtkit-daemon1134522-21ssh12:22:252
190898100,3rtkit-daemon669626-21munin-node09:54:260
190898100,3rtkit-daemon1076288-21ssh11:45:440
190898100,3rtkit-daemon1053827-21ssh11:34:560
10932552100,1sleep21093263-21ssh11:56:522
924121290,7sleep0924142-21idleruntime-cro10:15:000
873998291,6sleep1874005-21sh09:41:071
806238291,5sleep0806390-21df_inode09:00:140
723436291,5sleep2723463-21grep07:55:142
656987290,4sleep00-21swapper/007:05:110
19089890,6rtkit-daemon667-21snmpd09:25:292
19089890,6rtkit-daemon473-21gmain08:20:581
19089890,5rtkit-daemon858528-21ssh09:32:470
19089890,5rtkit-daemon818735-21diskmemload11:10:410
19089890,5rtkit-daemon716907-21/usr/sbin/munin07:50:181
19089890,5rtkit-daemon667-21snmpd09:22:382
19089890,5rtkit-daemon624096-21munin-node07:09:371
19089890,4rtkit-daemon954545-21ssh10:30:382
19089890,4rtkit-daemon779818-21sed08:35:252
19089890,4rtkit-daemon779818-21sed08:35:252
19089890,4rtkit-daemon1068604-21ssh11:41:370
19089890,4rtkit-daemon1028210-21ssh11:16:531
19089890,2rtkit-daemon438-21wpa_supplicant09:55:520
1108765291,6sleep01108920-21timerwakeupswit12:05:270
1012311290,5sleep01012319-21ssh11:07:420
1005482292,4sleep11004998-21apt-get11:05:031
989585280,1sleep10-21swapper/110:55:161
900219280,4sleep1900262-21ssh10:00:141
859857281,4sleep1860020-21diskstats09:35:181
812317281,4sleep1812339-21apt09:05:001
735983280,4sleep1736016-21apt-get08:05:141
710555281,4sleep2710588-21apt-get07:45:122
704226281,3sleep1704259-21apt-get07:40:101
690077281,4sleep0690202-21python307:25:240
657189280,4sleep30-21swapper/307:05:143
19089880,5rtkit-daemon669626-21munin-node09:27:530
19089880,5rtkit-daemon667-21snmpd09:40:420
19089880,5rtkit-daemon473-21gmain11:51:292
19089880,5rtkit-daemon1928-21sendmail-mta07:49:321
19089880,4rtkit-daemon667-21snmpd10:01:040
19089880,4rtkit-daemon473-21gmain09:01:302
19089880,4rtkit-daemon438-21wpa_supplicant10:06:230
19089880,4rtkit-daemon438-21wpa_supplicant09:09:002
19089880,4rtkit-daemon1125633-21sh12:15:382
19089880,4rtkit-daemon1096449-21seq12:00:172
19089880,3rtkit-daemon955875-21ssh10:33:431
19089880,3rtkit-daemon786916-21apt-get08:45:142
19089880,1rtkit-daemon1907-21rtkit-daemon09:20:002
19089880,1rtkit-daemon0-21swapper/210:11:322
19089880,1rtkit-daemon0-21swapper/208:33:342
1153887280,6sleep11155895-21sh12:35:221
1111394281,3sleep11111429-21apt-get12:10:141
1093655281,4sleep11093663-21ssh11:57:431
1082499281,4sleep01083601-21memory11:50:210
1062252280,2sleep11062260-21ssh11:40:121
1061592280,5sleep21061600-21ssh11:39:212
1046088281,3sleep11046127-21apt-get11:30:111
1037991281,4sleep20-21swapper/211:25:142
1019133280,5sleep21019410-21awk11:10:242
987104270,4sleep0987112-21ssh10:50:580
971421270,4sleep0971429-21ssh10:42:180
926971271,3sleep0929042-21kthreadcore10:15:200
913614270,1sleep10-21swapper/110:05:231
898862270,4sleep1898864-21ssh09:57:421
881834270,5sleep00-21swapper/009:45:310
857579270,5sleep1857585-21ssh09:30:411
843496270,4sleep10-21swapper/109:25:121
832572270,4sleep0832713-21/usr/sbin/munin09:15:250
793647270,5sleep1794305-21cut08:50:181
763750270,4sleep0765892-21cut08:25:210
717347270,4sleep0718189-21cut07:50:150
668554270,4sleep0668625-21ntp_states07:10:240
19089870,4rtkit-daemon671-21ntpd12:25:371
19089870,4rtkit-daemon669626-21munin-node10:45:091
19089870,4rtkit-daemon669626-21munin-node08:15:031
19089870,4rtkit-daemon669626-21munin-node07:28:141
19089870,4rtkit-daemon473-21gmain12:30:251
19089870,4rtkit-daemon473-21gmain10:35:101
19089870,3rtkit-daemon818735-21diskmemload10:50:441
19089870,3rtkit-daemon667-21snmpd10:55:142
19089870,3rtkit-daemon667-21snmpd08:13:501
19089870,3rtkit-daemon667-21snmpd07:20:121
19089870,3rtkit-daemon473-21gmain08:42:022
19089870,3rtkit-daemon438-21wpa_supplicant10:52:462
19089870,3rtkit-daemon1223-21cron08:54:012
19089870,3rtkit-daemon1135552-21ssh12:24:491
19089870,3rtkit-daemon1076597-21ssh11:46:292
19089870,1rtkit-daemon0-21swapper/107:58:121
1151198270,1sleep00-21swapper/012:33:280
1119730271,3sleep11119767-21apt-get12:15:141
1100308270,2sleep10-21swapper/112:00:231
1055624270,5sleep01057247-21/usr/sbin/munin11:35:180
1029886271,2sleep21029923-21apt-get11:20:132
1028689270,5sleep21028697-21ssh11:18:022
997223261,2sleep1997256-21apt-get11:00:131
970887260,1sleep20-21swapper/210:41:022
962571260,3sleep0962580-21ssh10:35:310
948676261,3sleep0948838-21diskstats10:30:140
948142260,1sleep10-21swapper/110:29:361
939706260,3sleep1939716-21ssh10:23:421
931767260,1sleep20-21swapper/210:19:022
891082260,1sleep10-21swapper/109:53:251
882275260,1sleep20-21swapper/209:46:372
875945260,1sleep10-21swapper/109:45:041
870034261,2sleep20-21swapper/209:40:192
836827260,3sleep0838068-21grep09:20:190
827171261,2sleep10-21swapper/109:15:141
817991260,3sleep1818125-21sed09:05:251
800658261,3sleep2801861-21cut08:55:202
799992260,3sleep1800657-21cut08:55:191
798512260,3sleep0798639-21sensors_temp08:50:290
769187260,3sleep0770359-21kthreadcore08:30:170
747698260,3sleep2747830-21sed08:10:242
737439260,3sleep2738953-21cut08:05:192
723669260,3sleep0724408-21irqrtprio07:55:160
704029260,3sleep0703705-21apt-get07:40:010
679210260,3sleep2680219-21cut07:20:162
19089860,3rtkit-daemon671-21ntpd12:32:402
19089860,3rtkit-daemon671-21ntpd08:25:151
19089860,3rtkit-daemon669626-21munin-node10:13:151
19089860,3rtkit-daemon669626-21munin-node07:30:131
19089860,3rtkit-daemon473-21gmain10:09:342
1153015261,2sleep00-21swapper/012:35:210
1125562260,1sleep00-21swapper/012:15:290
1109300260,1sleep10-21swapper/112:05:521
1094905260,1sleep00-21swapper/012:00:020
1061894260,4sleep10-21swapper/111:40:001
1046132261,3sleep21046198-21grep11:30:122
1035213260,3sleep01035330-21plymouth11:20:230
1021964261,3sleep01022086-21ssh11:15:140
1003272260,1sleep20-21swapper/211:01:022
962819250,3sleep2813549-21kworker/u16:4+rpciod10:36:092
899847250,3sleep20-21swapper/209:59:512
867124250,1sleep20-21swapper/209:38:592
858583250,3sleep2858591-21ssh09:32:552
825406250,3sleep10-21swapper/109:11:411
814300250,2sleep00-21swapper/009:05:190
799579250,3sleep00-21swapper/008:55:100
792877250,1sleep10-21swapper/108:48:461
792756250,1sleep00-21swapper/008:47:120
782000250,2sleep00-21swapper/008:40:200
780199250,1sleep00-21swapper/008:39:110
780199250,1sleep00-21swapper/008:39:110
776525251,2sleep10-21swapper/108:35:191
776525251,2sleep10-21swapper/108:35:191
773534250,1sleep10-21swapper/108:30:271
760866250,1sleep00-21swapper/008:20:520
749700250,3sleep0750572-21kthreadcore08:15:190
742358251,1sleep00-21swapper/008:10:140
735673250,1sleep20-21swapper/208:04:232
735646250,1sleep00-21swapper/008:04:010
722726250,1sleep20-21swapper/207:51:292
710003250,1sleep20-21swapper/207:41:112
703301250,1sleep00-21swapper/007:36:210
697470250,1sleep20-21swapper/207:35:072
690642250,1sleep20-21swapper/207:26:492
671396251,2sleep20-21swapper/207:15:032
662735250,1sleep20-21swapper/207:06:022
1141180250,3sleep21141263-21ntp_states12:25:212
1085596250,1sleep10-21swapper/111:53:261
1076646250,1sleep10-21swapper/111:46:381
1069723250,1sleep20-21swapper/211:44:182
978008240,3sleep00-21swapper/010:45:210
946828240,2sleep2946836-21ssh10:26:322
939104240,3sleep20-21swapper/210:22:192
757636240,2sleep20-21swapper/208:20:182
748918240,2sleep20-21swapper/208:15:182
673090240,2sleep0674197-21kthreadcore07:15:210
663355240,3sleep20-21swapper/207:10:142
1117247240,2sleep01117255-21ssh12:10:400
1012640240,3sleep20-21swapper/211:08:272
924161230,1sleep10-21swapper/110:15:011
780731230,1sleep10-21swapper/108:40:161
741308230,1sleep00-21swapper/008:05:260
691149230,1sleep00-21swapper/007:30:110
678429230,1sleep00-21swapper/007:20:140
1135950230,1sleep00-21swapper/012:25:120
1133135230,1sleep00-21swapper/012:20:220
1038091230,1sleep00-21swapper/011:25:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional