You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-10 - 07:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack8slot2s.osadl.org (updated Sun May 10, 2026 00:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1006096997365,5cyclictest86-21hwrng22:26:013
1006096997365,5cyclictest86-21hwrng22:26:013
1006095997265,4cyclictest0-21swapper/220:45:402
1006094997064,3cyclictest0-21swapper/100:32:591
1006095996964,3cyclictest0-21swapper/223:09:012
1006094996962,4cyclictest0-21swapper/123:04:551
1006094996962,4cyclictest0-21swapper/123:04:551
1006095996862,3cyclictest0-21swapper/200:20:422
1006094996862,3cyclictest0-21swapper/123:10:031
1006095996660,3cyclictest0-21swapper/200:31:582
1006095996660,3cyclictest0-21swapper/200:29:552
1006096996456,5cyclictest86-21hwrng00:00:133
1006096996456,5cyclictest86-21hwrng00:00:133
100609699642,19cyclictest1105359-21ssh22:40:143
1006095996458,3cyclictest0-21swapper/200:18:392
1006096996335,14cyclictest0-21swapper/323:15:543
100609699624,19cyclictest1064794-21ssh21:40:253
1006094996255,3cyclictest0-21swapper/100:16:361
1006096996018,37cyclictest1123240-21ssh23:05:523
1006096996017,13cyclictest1021353-21ntpq19:55:453
1006096995918,37cyclictest0-21swapper/322:21:063
1006096995918,15cyclictest1068287-21ssh21:45:273
1006096995917,38cyclictest0-21swapper/321:55:413
1006096995917,23cyclictest1109781-21ssh22:45:553
1006096995917,23cyclictest1109781-21ssh22:45:553
100609699591,21cyclictest1112131-21rm22:50:173
100609699582,21cyclictest1079267-21ssh22:00:513
100609699582,21cyclictest1079267-21ssh22:00:513
1006096995818,21cyclictest1163653-21ntpq00:05:503
1006096995816,36cyclictest0-21swapper/319:45:273
1006096995813,21cyclictest1136772-21ssh23:25:523
1006096995813,21cyclictest1136772-21ssh23:25:523
100609699581,28cyclictest1139421-21df23:30:273
100609699573,17cyclictest0-21swapper/300:10:133
100609699572,24cyclictest1058310-21ssh21:30:313
1006096995719,34cyclictest0-21swapper/322:35:093
1006096995719,34cyclictest0-21swapper/322:35:093
1006096995716,38cyclictest1183687-21ntpq00:35:543
1006096995716,35cyclictest0-21swapper/322:10:263
1006096995716,26cyclictest1033009-21kworker/u16:12+rpciod21:15:103
1006096995716,10cyclictest0-21swapper/321:11:053
100609699571,24cyclictest1098538-21sh22:30:103
1006096995712,10cyclictest0-21swapper/319:50:273
100609699568,14cyclictest1035-21CodeMeterLin19:15:083
1006096995618,21cyclictest1143124-21irqstats23:35:333
1006096995618,21cyclictest1143124-21irqstats23:35:333
1006096995614,10cyclictest0-21swapper/321:05:293
100609699561,20cyclictest1171372-21ssh00:17:073
100609699561,19cyclictest1145880-21apt-get23:40:093
100609699554,17cyclictest1121449-21systemd-userwor23:03:473
100609699554,17cyclictest1121449-21systemd-userwor23:03:473
100609699553,48cyclictest1132879-21grep23:20:353
100609699552,29cyclictest1054601-21ssh21:25:083
100609699552,21cyclictest1181678-21ssh00:33:063
1006096995517,32cyclictest0-21swapper/320:15:463
1006096995517,32cyclictest0-21swapper/320:15:463
1006096995516,9cyclictest0-21swapper/323:54:593
1006096995516,34cyclictest0-21swapper/320:50:343
1006096995515,8cyclictest0-21swapper/319:25:573
1006096995514,8cyclictest0-21swapper/321:20:433
1006096995513,9cyclictest0-21swapper/320:30:313
100609699551,35cyclictest1081701-21ssh22:05:173
100609699551,30cyclictest1015628-21modprobe19:39:593
100609699551,21cyclictest1149782-21/usr/sbin/munin23:45:333
100609699551,21cyclictest1090536-21ssh22:17:273
100609699551,21cyclictest1090536-21ssh22:17:273
1006096995420,15cyclictest1156560-21ssh23:55:343
1006096995418,16cyclictest1061649-21/usr/sbin/munin21:35:263
1006096995417,8cyclictest0-21swapper/320:20:293
1006096995417,17cyclictest1177057-21ssh00:25:533
1006096995416,4cyclictest0-21swapper/320:40:233
1006096995416,4cyclictest0-21swapper/320:40:233
1006096995416,4cyclictest0-21swapper/300:20:343
1006096995416,35cyclictest0-21swapper/323:10:253
100609699541,35cyclictest1072356-21ntpq21:50:393
100609699541,35cyclictest1072356-21ntpq21:50:393
1006096995319,31cyclictest0-21swapper/320:00:383
1006096995319,20cyclictest1115458-21ssh22:55:143
1006096995318,32cyclictest0-21swapper/320:08:263
1006096995314,13cyclictest1009296-21sendmail-msp19:20:033
1006096995313,7cyclictest0-21swapper/319:30:563
100609699531,22cyclictest1016083-21diskstats19:40:263
100609699531,21cyclictest1039579-21/usr/sbin/munin20:56:003
1006096995310,9cyclictest0-21swapper/321:00:313
1006096995310,9cyclictest0-21swapper/321:00:313
1006094995347,3cyclictest0-21swapper/120:44:381
1006094995347,3cyclictest0-21swapper/120:44:381
1006096995215,17cyclictest1037786-21apt-get20:49:593
1006096995210,12cyclictest0-21swapper/320:25:373
1006096995210,12cyclictest0-21swapper/320:25:373
1006096995110,20cyclictest1025322-21/usr/sbin/munin20:10:093
100609699491,17cyclictest1773-21snmpd19:10:533
100609699491,17cyclictest1773-21snmpd19:10:533
1006094994943,3cyclictest0-21swapper/120:46:411
100609399491,37cyclictest1089609-21/usr/sbin/munin22:16:020
100609399491,37cyclictest1089609-21/usr/sbin/munin22:16:020
1006093994913,23cyclictest1172759-21ssh00:20:220
100609699481,5cyclictest1033581-21grep20:35:283
1006093994822,22cyclictest1166019-21ssh00:10:060
100609399471,36cyclictest1109836-21ssh22:45:590
100609399471,36cyclictest1109836-21ssh22:45:590
100609399461,40cyclictest1055554-21ssh21:25:430
100609699452,14cyclictest1006229-21latency_hist19:10:003
100609699452,14cyclictest1006229-21latency_hist19:10:003
100609399452,28cyclictest1115468-21ssh22:55:170
1006093994522,18cyclictest1156878-21ssh23:55:470
1006093994519,22cyclictest1112183-21ssh22:50:270
100609399451,15cyclictest1170474-21ssh00:16:070
1006095994438,3cyclictest0-21swapper/223:04:552
1006095994438,3cyclictest0-21swapper/223:04:552
1006093994421,18cyclictest1163005-21ssh00:05:290
1006093994416,24cyclictest1125562-21ssh23:10:090
1006093994415,24cyclictest1133274-21ssh23:20:460
1006093994412,22cyclictest1098601-21ssh22:30:220
100609399441,20cyclictest1106309-21ntpq22:40:470
1006094994337,3cyclictest0-21swapper/121:50:161
1006094994337,3cyclictest0-21swapper/121:50:161
1006093994320,19cyclictest1773-21snmpd19:25:470
1006093994320,19cyclictest1182672-21ssh00:35:200
1006093994319,19cyclictest1176045-21ssh00:25:190
1006093994316,17cyclictest1033009-21kworker/u16:12+rpciod21:50:150
1006093994316,17cyclictest1033009-21kworker/u16:12+rpciod21:50:150
100609399431,38cyclictest1092869-21ssh22:20:560
100609399431,32cyclictest1051258-21ssh21:20:160
100609399431,15cyclictest1152640-21ssh23:50:220
1006095994238,1cyclictest0-21swapper/223:10:032
1006093994219,19cyclictest1123133-21ssh23:05:450
1006093994218,20cyclictest1064759-21ssh21:40:170
1006093994218,19cyclictest1135768-21ssh23:25:190
1006093994218,19cyclictest1135768-21ssh23:25:190
1006093994217,21cyclictest1147079-21ssh23:41:200
1006093994216,21cyclictest1048834-21ntpq21:15:500
100609399420,37cyclictest1096075-21ssh22:25:400
100609399420,37cyclictest1096075-21ssh22:25:400
1006093994120,17cyclictest1069106-21ntpq21:45:510
1006093994119,17cyclictest1159324-21ssh00:00:190
1006093994119,17cyclictest1159324-21ssh00:00:190
1006093994118,19cyclictest1150090-21ssh23:45:430
1006093994118,15cyclictest1076524-21ssh21:56:140
1006093994117,20cyclictest1082669-21ssh22:05:520
1006093994117,20cyclictest1043588-21ld21:05:490
100609399411,36cyclictest1179387-21ssh00:30:210
100609399411,34cyclictest1039879-21fschecks_count20:55:300
100609399411,30cyclictest1038294-21fschecks_count20:50:300
100609399411,18cyclictest1079277-21ntpq22:00:520
100609399411,18cyclictest1079277-21ntpq22:00:520
1006094994034,3cyclictest0-21swapper/122:53:401
100609399401,19cyclictest1041740-21cat21:00:320
100609399401,19cyclictest1041740-21cat21:00:320
100609399401,18cyclictest1118838-21ssh23:00:200
100609399401,18cyclictest1118838-21ssh23:00:200
100609399392,15cyclictest1139890-21sed23:30:370
1006093993919,17cyclictest1031130-21/usr/sbin/munin20:25:570
1006093993919,17cyclictest1031130-21/usr/sbin/munin20:25:570
1006093993918,17cyclictest1086123-21ssh22:10:590
1006093993918,17cyclictest1015552-21users19:35:590
1006093993917,18cyclictest1143411-21ntpq23:35:440
1006093993917,18cyclictest1143411-21ntpq23:35:440
1006093993917,18cyclictest1129003-21ssh23:15:210
1006093993917,17cyclictest1018137-21sed19:45:360
1006093993916,19cyclictest1102004-21ssh22:35:210
1006093993916,19cyclictest1102004-21ssh22:35:210
100609399391,33cyclictest1021880-21users19:56:000
100609399391,29cyclictest1044147-21diskmemload21:10:440
100609399391,18cyclictest1007019-21/usr/sbin/munin19:10:440
100609399391,18cyclictest1007019-21/usr/sbin/munin19:10:440
100609399391,17cyclictest100327650kworker/u16:1+xprtiod19:15:090
100609399391,14cyclictest1061336-21ssh21:35:090
1006093993818,16cyclictest1036412-21/usr/sbin/munin20:45:330
1006093993817,17cyclictest1016690-21ntpq19:40:510
1006093993816,11cyclictest1058395-21/usr/sbin/munin21:30:350
1006093993720,12cyclictest1013992-21dropbear19:31:410
1006093993718,9cyclictest819115-21kworker/u16:3+rpciod20:10:010
1006093993718,15cyclictest1034154-21/usr/sbin/munin20:35:590
1006093993716,17cyclictest1022967-21ntpq20:00:440
1006093993714,18cyclictest1773-21snmpd20:20:590
100609399371,32cyclictest1025564-21df_inode20:10:260
1006093993711,23cyclictest1010356-21ntp_states19:20:530
1006095993630,3cyclictest0-21swapper/219:14:582
1006095993630,3cyclictest0-21swapper/219:14:582
1006095993628,4cyclictest0-21swapper/221:20:202
1006094993629,4cyclictest0-21swapper/100:05:511
100609399365,27cyclictest1035375-21/usr/sbin/munin20:40:320
100609399365,27cyclictest1035375-21/usr/sbin/munin20:40:320
100609399361,14cyclictest1027972-21threads20:15:590
100609399361,14cyclictest1027972-21threads20:15:590
1006094993527,5cyclictest0-21swapper/122:00:371
1006094993527,5cyclictest0-21swapper/122:00:371
1006093993518,13cyclictest1019793-21ntpq19:50:450
1006095993428,3cyclictest0-21swapper/223:25:262
1006095993428,3cyclictest0-21swapper/223:25:262
1006095993428,3cyclictest0-21swapper/223:15:412
1006095993427,4cyclictest0-21swapper/223:36:032
1006095993427,4cyclictest0-21swapper/223:36:032
1006094993422,6cyclictest0-21swapper/123:46:001
100609399341,29cyclictest1032476-21ntpq20:30:460
1006094993327,3cyclictest0-21swapper/121:17:371
1006094993326,4cyclictest0-21swapper/121:40:551
1006094993325,5cyclictest0-21swapper/122:10:401
1006095993226,3cyclictest0-21swapper/222:20:052
1006095993225,4cyclictest0-21swapper/222:39:432
1006095993225,4cyclictest0-21swapper/222:39:432
1006094993225,4cyclictest0-21swapper/122:45:121
1006094993225,4cyclictest0-21swapper/122:45:121
1006094993225,4cyclictest0-21swapper/119:21:391
1006094993222,6cyclictest0-21swapper/123:40:241
1006095993125,3cyclictest0-21swapper/223:45:182
1006095993125,3cyclictest0-21swapper/221:36:172
1006095993125,3cyclictest0-21swapper/220:28:152
1006095993125,3cyclictest0-21swapper/220:28:152
1006095993124,4cyclictest0-21swapper/221:15:162
1006094993123,5cyclictest0-21swapper/123:40:001
1006094993123,5cyclictest0-21swapper/123:40:001
1006094993123,5cyclictest0-21swapper/121:35:301
1006094993123,4cyclictest0-21swapper/122:40:061
1006095993024,3cyclictest0-21swapper/223:56:072
1006095993024,3cyclictest0-21swapper/221:40:582
1006095993024,3cyclictest0-21swapper/221:10:322
1006095993023,4cyclictest0-21swapper/222:56:062
1006095993023,4cyclictest0-21swapper/220:00:212
1006094993025,3cyclictest0-21swapper/119:55:131
1006094993024,3cyclictest0-21swapper/121:00:011
1006094993024,3cyclictest0-21swapper/100:29:221
1006094993023,4cyclictest0-21swapper/122:26:011
1006094993023,4cyclictest0-21swapper/122:26:011
1006094993023,4cyclictest0-21swapper/121:20:291
1006094993021,5cyclictest0-21swapper/119:35:571
1006095992923,3cyclictest0-21swapper/222:45:482
1006095992923,3cyclictest0-21swapper/222:45:482
1006095992923,3cyclictest0-21swapper/222:05:202
1006095992923,3cyclictest0-21swapper/220:40:242
1006095992923,3cyclictest0-21swapper/220:40:242
1006095992923,3cyclictest0-21swapper/220:15:302
1006095992923,3cyclictest0-21swapper/220:15:302
1006095992923,3cyclictest0-21swapper/220:05:082
1006095992922,4cyclictest0-21swapper/221:50:172
1006095992922,4cyclictest0-21swapper/221:50:172
1006095992922,4cyclictest0-21swapper/200:36:032
1006095992922,4cyclictest0-21swapper/200:14:592
1006094992924,3cyclictest0-21swapper/121:02:031
1006094992924,3cyclictest0-21swapper/121:02:031
1006094992923,3cyclictest0-21swapper/122:20:021
1006094992923,3cyclictest0-21swapper/120:05:491
1006094992923,3cyclictest0-21swapper/100:00:031
1006094992923,3cyclictest0-21swapper/100:00:031
1006094992921,5cyclictest0-21swapper/122:15:441
1006094992921,5cyclictest0-21swapper/122:15:441
1006094992921,5cyclictest0-21swapper/100:20:211
1006094992921,4cyclictest0-21swapper/121:55:341
1006094992920,6cyclictest0-21swapper/123:25:281
1006094992920,6cyclictest0-21swapper/123:25:281
1006094992920,5cyclictest0-21swapper/123:55:591
1006094992920,5cyclictest0-21swapper/122:55:331
1006094992920,5cyclictest0-21swapper/122:30:421
1006094992920,5cyclictest0-21swapper/121:45:491
1006094992920,5cyclictest0-21swapper/121:10:421
1006094992919,7cyclictest0-21swapper/122:35:211
1006094992919,7cyclictest0-21swapper/122:35:211
1006094992919,5cyclictest0-21swapper/119:30:091
1006095992822,3cyclictest0-21swapper/222:50:142
1006095992822,3cyclictest0-21swapper/222:15:152
1006095992822,3cyclictest0-21swapper/222:15:152
1006095992822,3cyclictest0-21swapper/222:10:132
1006095992822,3cyclictest0-21swapper/222:00:302
1006095992822,3cyclictest0-21swapper/222:00:302
1006095992822,3cyclictest0-21swapper/221:01:262
1006095992822,3cyclictest0-21swapper/221:01:262
1006095992822,3cyclictest0-21swapper/220:50:102
1006095992822,3cyclictest0-21swapper/219:30:382
1006095992822,3cyclictest0-21swapper/219:20:292
1006095992822,3cyclictest0-21swapper/200:00:252
1006095992822,3cyclictest0-21swapper/200:00:252
1006095992821,4cyclictest0-21swapper/223:50:432
1006095992821,4cyclictest0-21swapper/223:30:572
1006095992821,4cyclictest0-21swapper/200:05:282
1006095992820,4cyclictest0-21swapper/223:40:222
1006095992820,4cyclictest0-21swapper/221:45:512
1006094992822,4cyclictest0-21swapper/100:35:401
1006094992822,3cyclictest0-21swapper/122:05:211
1006094992822,3cyclictest0-21swapper/121:05:471
1006094992822,3cyclictest0-21swapper/119:50:271
1006094992822,3cyclictest0-21swapper/119:25:181
1006094992821,4cyclictest0-21swapper/123:20:051
1006094992821,4cyclictest0-21swapper/121:25:241
1006094992821,4cyclictest0-21swapper/119:45:291
1006094992821,4cyclictest0-21swapper/100:10:331
1006094992820,5cyclictest0-21swapper/121:30:571
1006094992819,5cyclictest0-21swapper/123:05:461
1006094992819,5cyclictest0-21swapper/120:00:271
1006094992819,5cyclictest0-21swapper/119:40:281
1006094992817,7cyclictest0-21swapper/123:50:251
1006094992816,7cyclictest0-21swapper/120:50:101
100609399281,15cyclictest1006122-21latency_hist19:09:590
100609399281,15cyclictest1006122-21latency_hist19:09:590
1006095992722,3cyclictest0-21swapper/221:25:422
1006095992721,3cyclictest0-21swapper/223:21:062
1006095992721,3cyclictest0-21swapper/222:40:202
1006095992721,3cyclictest0-21swapper/222:32:192
1006095992721,3cyclictest0-21swapper/222:25:202
1006095992721,3cyclictest0-21swapper/222:25:202
1006095992721,3cyclictest0-21swapper/221:55:312
1006095992721,3cyclictest0-21swapper/221:05:492
1006095992721,3cyclictest0-21swapper/220:35:582
1006095992721,3cyclictest0-21swapper/219:40:232
1006095992721,3cyclictest0-21swapper/219:25:482
1006095992720,4cyclictest0-21swapper/219:50:322
1006095992719,4cyclictest0-21swapper/220:20:482
1006094992721,3cyclictest0-21swapper/123:32:571
1006094992721,3cyclictest0-21swapper/123:15:081
1006094992721,3cyclictest0-21swapper/120:25:321
1006094992721,3cyclictest0-21swapper/120:25:321
1006094992719,5cyclictest0-21swapper/120:10:341
1006094992719,4cyclictest0-21swapper/120:15:301
1006094992719,4cyclictest0-21swapper/120:15:301
1006095992620,3cyclictest0-21swapper/221:30:342
1006095992620,3cyclictest0-21swapper/220:55:322
1006095992620,3cyclictest0-21swapper/220:10:372
1006095992620,3cyclictest0-21swapper/219:55:542
1006095992620,3cyclictest0-21swapper/219:45:122
1006095992620,3cyclictest0-21swapper/219:35:482
1006095992620,3cyclictest0-21swapper/219:15:532
1006094992621,3cyclictest0-21swapper/120:30:251
1006094992620,3cyclictest0-21swapper/120:35:561
1006094992620,3cyclictest0-21swapper/119:15:141
1006094992619,4cyclictest0-21swapper/120:20:511
1006094992619,4cyclictest0-21swapper/119:10:461
1006094992619,4cyclictest0-21swapper/119:10:461
1006095992518,4cyclictest0-21swapper/220:30:462
1006095992115,3cyclictest0-21swapper/219:10:002
1006095992115,3cyclictest0-21swapper/219:10:002
1006094992115,4cyclictest0-21swapper/119:10:001
1006094992115,4cyclictest0-21swapper/119:10:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional