You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-04 - 05:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Thu Jun 04, 2026 00:42:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
958618995646,5cyclictest0-21swapper/200:21:012
958618995040,6cyclictest1036884-21sh22:41:052
958617995038,8cyclictest3408135-21cups-browsed23:51:211
958617994940,5cyclictest0-21swapper/122:35:001
958617994940,5cyclictest0-21swapper/122:35:001
958617994919,9cyclictest1106445-21ntpq00:31:051
958617994840,5cyclictest1054-21mta-sts-daemon23:03:411
958617994837,6cyclictest0-21swapper/123:26:021
958617994512,12cyclictest966589-21ntpq19:51:001
958617994415,8cyclictest1003716-21ssh21:49:261
958616994426,15cyclictest1070459-21latency_hist23:35:020
958618994338,2cyclictest0-21swapper/222:00:012
958617994336,4cyclictest0-21swapper/100:00:001
958617994316,7cyclictest1008403-21ntpq21:56:071
958617994315,8cyclictest995654-21ssh21:36:041
958617994314,8cyclictest1071510-21ntpq23:36:041
958617994314,8cyclictest1034553-21sh22:37:051
958617994312,13cyclictest1090290-21irqstats00:05:571
958616994336,4cyclictest1010254-21munin-run22:00:010
95861799423,28cyclictest1027329-21ssh22:26:021
95861799423,28cyclictest1027329-21ssh22:26:021
958617994226,12cyclictest271ktimers/119:11:031
958617994226,12cyclictest271ktimers/119:11:031
958617994215,8cyclictest1014762-21ntpq22:06:081
958617994215,7cyclictest1087369-21ntpq00:01:001
958617994214,11cyclictest1109602-21ntpq00:36:041
958617994212,11cyclictest972898-21cut20:30:021
95861899413,33cyclictest1062208-21ssh23:21:082
958618994132,6cyclictest1-21systemd22:05:012
958617994116,7cyclictest0-21swapper/122:05:011
958617994113,9cyclictest961229-21ntpq19:20:571
958617994113,8cyclictest1093294-21df00:10:541
958617994032,5cyclictest986754-21grep21:21:131
958617994029,8cyclictest1054566-21sh23:10:011
958617994016,8cyclictest998937-21ntpq21:41:071
958617994015,9cyclictest1048258-21sendmail-msp23:00:001
958617994015,9cyclictest1048258-21sendmail-msp23:00:001
958617994015,8cyclictest964782-21ntpq19:40:581
958617993931,5cyclictest976261-21idleruntime-cro20:50:001
958617993915,7cyclictest1100873-21ssh00:21:411
958617993914,9cyclictest963161-21processes19:31:051
958617993914,9cyclictest1079016-21sh23:47:431
958617993914,8cyclictest1102958-21irqstats00:25:571
958617993913,9cyclictest1096873-21ntpq00:16:021
958617993913,8cyclictest968289-21ntpq20:00:571
958617993910,12cyclictest1060737-21ssh23:19:451
958616993931,5cyclictest421-21dbus-daemon20:40:010
958616993931,5cyclictest421-21dbus-daemon20:40:010
958616993914,21cyclictest1011660-21ssh22:01:120
958618993832,3cyclictest0-21swapper/222:11:082
958618993832,3cyclictest0-21swapper/222:11:082
958618993831,3cyclictest0-21swapper/221:11:042
958618993830,5cyclictest1016575-21cron22:10:002
958618993830,3cyclictest0-21swapper/223:51:042
958618993829,5cyclictest3408135-21cups-browsed22:26:102
958618993829,5cyclictest3408135-21cups-browsed22:26:102
958618993829,5cyclictest1077466-21fschecks_count23:45:552
958617993830,4cyclictest0-21swapper/122:11:141
958617993830,4cyclictest0-21swapper/122:11:141
958617993827,7cyclictest0-21swapper/121:20:011
958617993815,7cyclictest980584-21/usr/sbin/munin21:10:521
958617993814,9cyclictest1040067-21ntpq22:46:061
958617993813,9cyclictest1042896-21latency22:50:591
958616993830,5cyclictest1059532-21unixbench_singl23:16:160
958616993821,12cyclictest12350irq/32-eth022:36:070
95861699381,33cyclictest1052587-21ssh23:06:010
958618993729,5cyclictest1073477-21idleruntime-cro23:40:012
958617993724,8cyclictest1062171-21ntpq23:21:061
958617993715,7cyclictest1055601-21memory23:11:021
958617993714,7cyclictest421-21dbus-daemon19:29:071
958617993713,7cyclictest988425-21sh21:25:231
958617993713,7cyclictest1067889-21cpu23:30:551
958617993712,8cyclictest1007104-21ssh21:55:011
958617993712,7cyclictest1074674-21collect223:41:011
958616993730,4cyclictest1103156-21needreboot00:26:000
958616993730,4cyclictest1048214-21ssh23:00:000
958616993730,4cyclictest1048214-21ssh23:00:000
958616993729,5cyclictest3408135-21cups-browsed23:50:480
958616993726,8cyclictest961896-21kthreadd19:25:010
95861699371,30cyclictest1027349-21rm22:26:030
95861699371,30cyclictest1027349-21rm22:26:030
958618993629,4cyclictest695-21cron23:30:012
958618993628,5cyclictest1046861-21grep22:56:132
958618993628,5cyclictest1046861-21grep22:56:132
958617993628,5cyclictest967419-21awk20:00:011
958617993626,7cyclictest0-21swapper/119:40:011
958617993624,8cyclictest973648-21ntpq20:31:001
958617993623,8cyclictest970058-21ntpq20:10:581
958617993623,7cyclictest975470-21ntpq20:41:021
958617993611,10cyclictest1024202-21collect222:21:051
958617993611,10cyclictest1024202-21collect222:21:051
958617993610,7cyclictest1020113-21ssh22:15:261
958616993628,5cyclictest1020406-21telnet22:15:540
958618993529,3cyclictest0-21swapper/220:30:012
958618993527,4cyclictest1000743-21ssh21:45:012
958618993526,6cyclictest978053-21systemd-run21:00:012
95861799359,8cyclictest1037486-21ssh22:41:201
958617993522,8cyclictest978916-21ntpq21:00:591
958617993519,9cyclictest977988-21sshd20:57:151
958617993512,7cyclictest960428-21date19:20:011
958617993512,7cyclictest960428-21date19:20:011
958616993529,3cyclictest0-21swapper/023:36:070
958616993528,5cyclictest1056-21mta-sts-daemon19:41:020
958616993522,10cyclictest0-21swapper/022:50:550
958618993428,3cyclictest0-21swapper/223:15:022
958618993427,3cyclictest0-21swapper/222:46:012
958618993426,5cyclictest967418-21munin-plugin-st20:00:012
958618993426,5cyclictest3408135-21cups-browsed21:20:252
958618993426,5cyclictest1074885-21proc_pri23:41:092
958618993426,4cyclictest0-21swapper/200:06:052
958618993425,6cyclictest3408135-21cups-browsed21:38:262
958618993423,8cyclictest967155-21idleruntime-cro19:55:012
958617993425,3cyclictest0-21swapper/121:10:031
958617993422,9cyclictest971841-21ntpq20:20:581
958617993422,7cyclictest977170-21ntpq20:51:001
95861699346,3cyclictest0-21swapper/020:11:110
958616993420,12cyclictest1056-21mta-sts-daemon23:41:560
958618993327,3cyclictest0-21swapper/220:20:012
958618993326,4cyclictest979505-21sshd21:01:162
958618993326,4cyclictest1024446-21mailstats22:21:122
958618993326,4cyclictest1024446-21mailstats22:21:122
958618993325,5cyclictest1102889-21sh00:25:572
958618993325,3cyclictest0-21swapper/221:26:072
95861799338,8cyclictest991608-21ssh21:30:261
958617993317,12cyclictest969202-21/usr/sbin/munin20:10:021
958616993327,4cyclictest1110178-21sed00:36:140
958616993327,4cyclictest1016573-21cron22:10:000
958616993326,4cyclictest1037343-21unixbench_multi22:41:130
958616993321,10cyclictest0-21swapper/019:35:020
958618993225,4cyclictest1086171-21sendmail00:00:012
958618993225,4cyclictest1056-21mta-sts-daemon19:29:082
958618993225,3cyclictest0-21swapper/223:05:522
958618993224,5cyclictest979797-21diskmemload00:10:582
958618993224,5cyclictest976019-21systemd-run20:45:012
958618993224,5cyclictest0-21swapper/220:40:002
958618993224,5cyclictest0-21swapper/220:40:002
958616993227,3cyclictest0-21swapper/019:20:000
958616993227,3cyclictest0-21swapper/019:20:000
958616993225,4cyclictest1018232-21timerandwakeup22:11:140
958616993225,4cyclictest1018232-21timerandwakeup22:11:140
958616993223,6cyclictest988632-21aten_r2power_cu21:25:520
958616993223,6cyclictest0-21swapper/021:10:020
958616993222,7cyclictest1051-21mta-sts-daemon20:31:030
95861699322,23cyclictest1040050-21ssh22:46:050
95861899319,11cyclictest1056-21mta-sts-daemon20:09:442
958618993125,3cyclictest0-21swapper/221:10:012
958618993124,4cyclictest993064-21unixbench_singl21:31:132
958618993123,5cyclictest1096737-21missed_timers00:16:002
958618993123,5cyclictest1055-21mta-sts-daemon22:31:252
958618993123,5cyclictest1055-21mta-sts-daemon22:31:252
958618993122,7cyclictest1087779-21threads00:01:122
958617993122,6cyclictest970940-21sshd20:18:161
958616993124,4cyclictest992069-21uniq21:30:550
958616993122,6cyclictest998094-21/usr/sbin/munin21:40:520
958616993118,10cyclictest968932-21cron20:05:010
958616993118,10cyclictest0-21swapper/021:25:020
958616993113,13cyclictest12350irq/32-eth000:06:060
958616993112,3cyclictest171rcu_preempt23:56:070
958618993024,3cyclictest0-21swapper/221:20:012
958618993023,4cyclictest1059468-21grep23:16:162
958618993023,4cyclictest1054-21mta-sts-daemon00:31:042
958618993022,5cyclictest1020736-21latency22:15:582
95861899302,21cyclictest1049685-21ps23:01:132
958617993022,5cyclictest974495-21basename20:40:001
958617993022,5cyclictest974495-21basename20:40:001
958616993025,3cyclictest3408135-21cups-browsed21:51:150
958616993024,3cyclictest0-21swapper/000:31:020
958616993022,6cyclictest1096455-21sh00:15:550
95861699302,22cyclictest1077873-21ssh23:46:020
958616993014,10cyclictest12350irq/32-eth022:31:070
958616993014,10cyclictest12350irq/32-eth022:31:070
958616993010,2cyclictest171rcu_preempt00:01:110
958618992922,4cyclictest1056-21mta-sts-daemon23:30:252
958618992921,5cyclictest420-21avahi-daemon00:37:562
95861699294,3cyclictest970944-21kworker/0:0+events_freezable_power_20:21:020
958616992923,4cyclictest1056-21mta-sts-daemon23:10:560
958616992923,3cyclictest0-21swapper/023:01:020
958616992922,4cyclictest1055-21mta-sts-daemon21:14:170
958616992919,7cyclictest0-21swapper/023:21:030
95861699291,24cyclictest995652-21rm21:36:040
95861899284,11cyclictest1043286-21ssh22:51:102
958618992821,4cyclictest963893-21grep19:40:012
958618992821,4cyclictest1055-21mta-sts-daemon19:10:022
958618992821,4cyclictest0-21swapper/220:15:012
958618992821,3cyclictest0-21swapper/222:36:042
958618992819,6cyclictest3408135-21cups-browsed20:01:052
958618992811,3cyclictest171rcu_preempt20:31:012
958617992815,7cyclictest3408135-21cups-browsed19:45:391
958616992823,3cyclictest0-21swapper/021:00:010
958616992816,4cyclictest979797-21diskmemload00:24:540
958618992719,4cyclictest0-21swapper/221:51:092
958616992721,4cyclictest984903-21sendmail21:20:010
958616992718,7cyclictest0-21swapper/019:15:020
958616992718,7cyclictest0-21swapper/019:15:020
958616992713,9cyclictest659-21snmpd00:11:450
958616992620,4cyclictest966122-21cpuspeed19:50:490
958616992620,4cyclictest1065614-21timerandwakeup23:26:120
958616992618,5cyclictest963928-21systemd-run19:40:010
958616992618,5cyclictest420-21avahi-daemon20:49:450
958616992611,10cyclictest12350irq/32-eth022:21:080
958616992611,10cyclictest12350irq/32-eth022:21:080
958618992520,2cyclictest0-21swapper/219:15:012
958618992520,2cyclictest0-21swapper/219:15:012
958618992519,2cyclictest0-21swapper/219:20:002
958618992519,2cyclictest0-21swapper/219:20:002
958618992514,8cyclictest1001958-21ntpq21:46:012
958618992514,4cyclictest967141-21kworker/u8:4-xprtiod20:51:002
958618992513,9cyclictest964742-21awk19:40:562
958617992515,7cyclictest958570-21munin-run19:10:011
95861699254,6cyclictest171rcu_preempt20:17:090
95861699254,11cyclictest14-21ksoftirqd/020:41:030
958616992520,3cyclictest1054-21mta-sts-daemon19:48:140
958616992519,3cyclictest1002006-21ntp_kernel_pll_21:46:020
958618992417,5cyclictest0-21swapper/219:50:032
958618992416,5cyclictest3408135-21cups-browsed20:23:522
95861699249,10cyclictest3408135-21cups-browsed21:00:570
958618992316,4cyclictest0-21swapper/219:30:592
958616992316,5cyclictest0-21swapper/020:00:010
958618992215,5cyclictest0-21swapper/219:21:122
958616992216,4cyclictest3408135-21cups-browsed20:53:370
958618992113,5cyclictest3408135-21cups-browsed20:47:332
95861699213,10cyclictest14-21ksoftirqd/019:30:000
958616992115,4cyclictest548-21gdbus20:07:090
958616992015,3cyclictest947423-21kworker/u8:16-rpciod19:10:010
958616992014,3cyclictest965393-21kworker/u8:7+rpciod20:30:010
95861999182,9cyclictest0-21swapper/300:11:053
95861999171,9cyclictest0-21swapper/300:06:013
95861999162,8cyclictest0-21swapper/323:56:053
95861999162,8cyclictest0-21swapper/321:51:033
95861999162,7cyclictest0-21swapper/321:26:023
95861999161,8cyclictest0-21swapper/300:26:033
95861999161,7cyclictest0-21swapper/322:01:083
95861999161,7cyclictest0-21swapper/321:31:003
95861999161,7cyclictest0-21swapper/320:11:043
95861999161,7cyclictest0-21swapper/300:16:033
95861999153,6cyclictest0-21swapper/322:31:073
95861999153,6cyclictest0-21swapper/322:31:073
95861999152,6cyclictest0-21swapper/300:21:023
95861999151,8cyclictest0-21swapper/322:46:063
95861999151,7cyclictest0-21swapper/321:21:033
95861999143,6cyclictest0-21swapper/321:56:083
95861999142,9cyclictest0-21swapper/323:15:543
95861999142,8cyclictest0-21swapper/322:11:053
95861999142,8cyclictest0-21swapper/322:11:053
95861999142,8cyclictest0-21swapper/300:31:003
95861999142,7cyclictest0-21swapper/323:51:063
95861999142,7cyclictest0-21swapper/321:11:033
95861999141,8cyclictest0-21swapper/323:21:013
95861999141,7cyclictest0-21swapper/323:11:043
95861999141,7cyclictest0-21swapper/322:26:003
95861999141,7cyclictest0-21swapper/322:26:003
95861999141,7cyclictest0-21swapper/321:00:573
95861999141,6cyclictest0-21swapper/320:20:583
95861999133,8cyclictest0-21swapper/319:30:583
95861999132,7cyclictest0-21swapper/323:05:523
95861999132,7cyclictest0-21swapper/322:45:023
95861999132,6cyclictest0-21swapper/323:01:023
95861999132,5cyclictest0-21swapper/323:30:533
95861999132,5cyclictest0-21swapper/320:31:003
95861999131,8cyclictest0-21swapper/323:46:003
95861999131,8cyclictest0-21swapper/321:45:133
95861999131,8cyclictest0-21swapper/321:40:513
95861999131,7cyclictest0-21swapper/321:36:013
95861999131,7cyclictest0-21swapper/321:20:013
95861999131,7cyclictest0-21swapper/319:40:583
95861999131,6cyclictest0-21swapper/320:41:013
95861999123,5cyclictest0-21swapper/322:06:033
95861999123,5cyclictest0-21swapper/300:36:013
95861999122,8cyclictest0-21swapper/323:36:003
95861999121,7cyclictest0-21swapper/323:41:023
95861999121,7cyclictest0-21swapper/322:51:073
95861999121,7cyclictest0-21swapper/300:01:033
95861999121,6cyclictest0-21swapper/321:10:013
95861999121,5cyclictest0-21swapper/320:50:573
95861999121,5cyclictest0-21swapper/320:10:023
95861999120,6cyclictest0-21swapper/319:50:583
95861999112,6cyclictest0-21swapper/320:40:003
95861999112,6cyclictest0-21swapper/320:40:003
95861999112,5cyclictest0-21swapper/320:20:013
95861999112,5cyclictest0-21swapper/319:40:013
95861999111,7cyclictest0-21swapper/323:25:503
95861999111,7cyclictest0-21swapper/322:55:513
95861999111,7cyclictest0-21swapper/322:55:513
95861999111,7cyclictest0-21swapper/322:35:593
95861999111,6cyclictest0-21swapper/319:20:003
95861999111,6cyclictest0-21swapper/319:20:003
95861999111,6cyclictest0-21swapper/319:11:023
95861999111,6cyclictest0-21swapper/319:11:023
95861999111,5cyclictest0-21swapper/322:20:163
95861999111,5cyclictest0-21swapper/322:20:163
95861999111,5cyclictest0-21swapper/320:50:003
95861999102,6cyclictest0-21swapper/322:16:033
95861999100,5cyclictest0-21swapper/320:57:283
9586199993,4cyclictest0-21swapper/320:00:013
9586199992,6cyclictest0-21swapper/319:10:023
9586199992,4cyclictest0-21swapper/320:00:563
9586199991,5cyclictest0-21swapper/319:50:013
9586199991,5cyclictest0-21swapper/319:20:493
9586199981,5cyclictest0-21swapper/320:30:023
9586199981,4cyclictest0-21swapper/319:30:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional