You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-01 - 21:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Sat Aug 01, 2026 12:45:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
97206599904,62cyclictest1042088-21ssh11:25:021
97206099771,73cyclictest0-21swapper/011:05:020
972060996935,14cyclictest985326-21ssh09:25:000
97206099661,39cyclictest1064755-21kworker/u8:212:15:020
97206099661,39cyclictest1064755-21kworker/u8:212:15:020
97206599653,56cyclictest978399-21ssh09:10:021
97206599653,56cyclictest978399-21ssh09:10:021
972065996523,38cyclictest1054818-21sh11:52:011
97206599651,14cyclictest1073504-21sshd12:32:031
97206599651,14cyclictest1073504-21sshd12:32:021
97206599632,16cyclictest977217-21/usr/sbin/munin08:50:021
972065996320,37cyclictest978239-21diskmemload12:17:291
97206599631,9cyclictest1015491-21ssh10:28:491
97206599631,9cyclictest1015491-21ssh10:28:491
972065996318,41cyclictest23-21ksoftirqd/111:03:181
97206599631,6cyclictest976952-21munin-run08:45:011
97206599631,6cyclictest976952-21munin-run08:45:011
97206099639,50cyclictest16950irq/224-800000011:35:050
972060996339,16cyclictest275-21dbus-daemon11:55:010
972065996229,30cyclictest23-21ksoftirqd/111:38:131
97206599622,7cyclictest1043060-21ssh11:27:001
97206599621,8cyclictest1046553-21ssh11:34:421
97206599621,6cyclictest993558-21ssh09:42:121
97206599621,6cyclictest1001214-21ssh09:58:331
97206599621,10cyclictest1024227-21ssh10:47:141
97207099611,3cyclictest0-21swapper/207:55:012
972065996131,27cyclictest23-21ksoftirqd/111:41:311
97206599612,6cyclictest986079-21ssh09:26:141
97206599612,6cyclictest986079-21ssh09:26:141
97206599612,5cyclictest977468-21munin-run08:55:011
97206599612,14cyclictest1006656-21munin-run10:10:021
97206599611,9cyclictest1016233-21ssh10:30:051
97206599611,9cyclictest1008032-21ssh10:12:501
97206599611,8cyclictest980303-21ssh09:14:151
97206599611,8cyclictest1076887-21ssh12:39:251
97206599611,8cyclictest1030010-21ssh10:59:421
97206599611,8cyclictest1019885-21ssh10:38:051
97206599611,8cyclictest1019885-21ssh10:38:051
97206599611,8cyclictest1012005-21ssh10:21:031
97206599611,7cyclictest1035954-21ssh11:12:051
972065996117,40cyclictest23-21ksoftirqd/110:42:461
972065996112,45cyclictest23-21ksoftirqd/112:21:071
97206599611,11cyclictest3467602-21systemd-journal09:35:451
97206599611,10cyclictest1051970-21ssh11:45:531
97206099611,43cyclictest1071881-21ssh12:28:480
97206599601,9cyclictest1033088-21ssh11:05:531
97206599601,8cyclictest989538-21ssh09:33:481
97206599601,8cyclictest981230-21ssh09:15:511
97206599601,8cyclictest1056773-21ssh11:55:591
97206599601,8cyclictest1037885-21ssh11:15:581
97206599601,7cyclictest998983-21ssh09:53:431
97206599601,7cyclictest995305-21ssh09:45:441
97206599601,7cyclictest1070396-21ssh12:25:061
97206599601,7cyclictest1063600-21ssh12:10:461
97206599601,7cyclictest1063600-21ssh12:10:451
97206599601,7cyclictest1027115-21ssh10:53:291
97206599601,5cyclictest976180-21munin-run08:30:021
97206599601,5cyclictest976180-21munin-run08:30:011
97206099602,48cyclictest1063031-21munin-plugin-st12:10:000
97206099602,18cyclictest987733-21latency_hist09:30:010
97206099602,18cyclictest987733-21latency_hist09:30:010
97206599592,6cyclictest1060907-21ssh12:05:031
97206599592,5cyclictest364-21snmpd10:18:071
97206599591,7cyclictest1059386-21ssh12:02:001
97206599591,6cyclictest1002214-21ssh10:00:141
97206599591,6cyclictest1002214-21ssh10:00:141
97206599582,11cyclictest972802-21munin-run07:25:001
97206599581,7cyclictest984202-21ssh09:22:131
97206599581,6cyclictest364-21snmpd07:42:391
972060995827,15cyclictest993491-21ssh09:42:030
972060995729,13cyclictest1041774-21ssh11:24:250
972060995728,14cyclictest1022281-21ssh10:43:080
97206599562,5cyclictest974342-21munin-run07:55:001
972060995646,7cyclictest12-21ksoftirqd/008:35:020
972060995630,11cyclictest1013916-21ssh10:25:070
972060995630,11cyclictest1013916-21ssh10:25:070
972060995627,14cyclictest1028501-21ssh10:56:030
972070995546,5cyclictest974351-21kworker/2:111:36:532
972070995545,6cyclictest1018375-21kworker/2:010:55:022
97206599552,13cyclictest976002-21cat08:25:011
972065995520,31cyclictest973317-21systemd-run07:35:011
972065995520,31cyclictest973317-21systemd-run07:35:011
972060995548,5cyclictest12-21ksoftirqd/008:25:020
972060995528,12cyclictest1038696-21ssh11:17:500
972060995527,13cyclictest980555-21ssh09:14:560
972060995527,12cyclictest1075544-21ssh12:36:140
972060995527,12cyclictest1033037-21ssh11:05:340
972060995526,13cyclictest1043358-21ssh11:27:450
972060995526,13cyclictest1024831-21ssh10:48:390
972060995523,28cyclictest12-21ksoftirqd/010:55:010
972070995446,5cyclictest977466-21kworker/2:309:47:132
972070995442,7cyclictest30-21ksoftirqd/211:18:402
97206599542,5cyclictest364-21snmpd08:57:521
97206599541,6cyclictest974877-21munin-run08:05:011
97206599541,14cyclictest974559-21sshd07:55:191
97206099547,28cyclictest1008984-21kworker/u8:210:15:030
972060995427,11cyclictest1035861-21ssh11:11:410
972060995425,13cyclictest1066761-21ssh12:17:470
972071995331,18cyclictest275-21dbus-daemon12:10:003
972070995344,6cyclictest974351-21kworker/2:111:25:522
972070995335,12cyclictest275-21dbus-daemon12:00:002
97206599532,6cyclictest364-21snmpd07:15:051
97206099537,34cyclictest12-21ksoftirqd/007:55:020
972060995333,4cyclictest12-21ksoftirqd/010:11:420
972060995332,16cyclictest1060742-21ssh12:05:020
972060995323,12cyclictest983081-21ssh09:20:020
97207099525,44cyclictest977087-21modprobe08:45:022
97207099525,44cyclictest977087-21modprobe08:45:022
972070995244,5cyclictest974351-21kworker/2:111:54:532
972070995244,5cyclictest974351-21kworker/2:111:09:072
972070995217,6cyclictest131rcu_preempt11:04:512
97206599523,8cyclictest275-21dbus-daemon09:05:001
97206599522,4cyclictest0-21swapper/108:16:521
97206599522,13cyclictest973024-21sshd07:26:031
97206599521,13cyclictest364-21snmpd08:37:001
972060995230,18cyclictest1004438-21sshd10:05:030
972060995230,18cyclictest1004438-21sshd10:05:020
972060995224,13cyclictest1017773-21ssh10:33:370
972060995222,14cyclictest1046725-21ssh11:35:010
97206099522,19cyclictest1001877-21munin-plugin-st10:00:000
97206099521,42cyclictest10123401ssh10:21:570
972070995143,5cyclictest977466-21kworker/2:309:35:092
972070995143,5cyclictest1039629-21kworker/2:212:39:082
972070995143,5cyclictest1039629-21kworker/2:212:10:012
972070995142,6cyclictest977466-21kworker/2:310:07:062
97206599512,4cyclictest364-21snmpd08:12:431
972065995120,26cyclictest309-21in:imuxsock07:15:011
972065995120,26cyclictest309-21in:imuxsock07:15:011
972060995142,7cyclictest12-21ksoftirqd/011:46:120
972060995133,13cyclictest1-21systemd07:45:010
972060995122,13cyclictest1056904-21ssh11:56:230
972060995116,31cyclictest12-21ksoftirqd/012:22:560
97207099504,31cyclictest965949-21kworker/u8:207:40:012
972070995042,5cyclictest977466-21kworker/2:309:58:162
972070995042,5cyclictest977466-21kworker/2:309:23:542
972070995042,5cyclictest974351-21kworker/2:112:02:182
972070995042,5cyclictest1039629-21kworker/2:212:29:522
972070995042,5cyclictest1018375-21kworker/2:010:57:442
972070995041,5cyclictest972825-21kworker/2:307:50:012
972070995041,5cyclictest972825-21kworker/2:307:50:012
97206599503,9cyclictest972068-21grep07:10:011
972065995021,24cyclictest23-21ksoftirqd/107:40:011
97206099501,7cyclictest151rcuc/011:45:020
97207099495,40cyclictest1016168-21diskmemload10:30:032
97207099495,40cyclictest1016168-21diskmemload10:30:022
972070994941,5cyclictest976433-21kworker/2:209:13:132
972070994941,5cyclictest974351-21kworker/2:110:31:142
972070994941,5cyclictest1039629-21kworker/2:212:13:302
972070994941,5cyclictest1039629-21kworker/2:212:13:292
972070994933,11cyclictest30-21ksoftirqd/210:22:022
972070994932,13cyclictest3467602-21systemd-journal09:30:022
97206599492,4cyclictest0-21swapper/108:34:411
97206599491,11cyclictest364-21snmpd07:47:401
97206599491,11cyclictest364-21snmpd07:47:401
97206099491,45cyclictest1005617-21ssh10:07:430
972071994833,12cyclictest3467602-21systemd-journal11:51:023
972071994831,13cyclictest3467602-21systemd-journal07:35:023
972071994831,13cyclictest3467602-21systemd-journal07:35:023
972070994840,5cyclictest974351-21kworker/2:112:20:122
972070994840,5cyclictest974351-21kworker/2:110:36:362
972070994840,5cyclictest974351-21kworker/2:110:36:352
972070994839,6cyclictest974351-21kworker/2:111:34:252
97206599481,4cyclictest1-21systemd08:10:001
972060994835,8cyclictest12-21ksoftirqd/007:15:020
972060994835,8cyclictest12-21ksoftirqd/007:15:010
972060994822,11cyclictest998813-21ssh09:53:230
972060994821,12cyclictest1020684-21ssh10:39:570
972060994821,12cyclictest1020684-21ssh10:39:570
97207199471,41cyclictest3467602-21systemd-journal09:25:003
972070994740,4cyclictest974351-21kworker/2:110:12:122
972070994739,5cyclictest974351-21kworker/2:112:19:032
972070994739,5cyclictest1018375-21kworker/2:010:46:102
972070994732,11cyclictest30-21ksoftirqd/209:20:002
972070994732,11cyclictest275-21dbus-daemon07:55:482
972060994740,4cyclictest16950irq/224-800000007:40:000
972060994733,7cyclictest978223-21cron09:10:000
972060994733,7cyclictest978223-21cron09:10:000
97206099471,4cyclictest131rcu_preempt09:45:220
972060994711,7cyclictest12-21ksoftirqd/009:00:010
972070994639,4cyclictest974351-21kworker/2:111:10:562
972070994633,8cyclictest30-21ksoftirqd/208:55:432
972060994633,9cyclictest12-21ksoftirqd/009:40:010
972060994630,12cyclictest976828-21date08:40:020
972060994630,12cyclictest1-21systemd08:05:010
97206099462,13cyclictest988240-21sshd09:30:460
972071994530,10cyclictest976674-21cron08:40:003
972071994528,13cyclictest275-21dbus-daemon09:40:013
972071994528,13cyclictest0-21swapper/310:35:003
97207199452,5cyclictest131rcu_preempt12:25:003
972070994538,4cyclictest977466-21kworker/2:309:25:452
972070994538,4cyclictest977466-21kworker/2:309:25:442
972070994537,5cyclictest974351-21kworker/2:112:31:252
972070994537,5cyclictest974351-21kworker/2:112:31:252
972070994532,10cyclictest30-21ksoftirqd/210:16:012
97206099452,38cyclictest976925-21sshd08:42:020
97206099452,38cyclictest976925-21sshd08:42:020
972070994436,5cyclictest974351-21kworker/2:111:41:302
972070994436,5cyclictest974351-21kworker/2:111:21:012
972070994436,5cyclictest971789-21kworker/2:007:32:342
972070994436,5cyclictest971789-21kworker/2:007:32:332
972070994434,6cyclictest974351-21kworker/2:110:41:082
97206099444,10cyclictest12-21ksoftirqd/008:10:020
972060994435,5cyclictest972548-21sendmail-msp07:20:000
972060994429,12cyclictest1074301-21ssh12:33:520
972060994429,12cyclictest1074301-21ssh12:33:520
972060994423,14cyclictest275-21dbus-daemon08:10:000
972071994330,10cyclictest37-21ksoftirqd/308:05:033
972071994329,9cyclictest3467602-21systemd-journal07:40:003
972071994328,12cyclictest1-21systemd11:00:013
972071994327,13cyclictest3467602-21systemd-journal08:06:023
97207199432,6cyclictest131rcu_preempt10:50:003
972070994335,5cyclictest977466-21kworker/2:310:01:172
972070994335,5cyclictest977466-21kworker/2:310:01:172
972070994333,5cyclictest977466-21kworker/2:309:54:562
972070994333,5cyclictest971789-21kworker/2:007:15:012
972070994333,5cyclictest971789-21kworker/2:007:15:012
972070994329,10cyclictest3467602-21systemd-journal09:05:032
972070994329,10cyclictest3467602-21systemd-journal09:05:022
972070994318,19cyclictest0-21swapper/208:25:012
972060994330,9cyclictest12-21ksoftirqd/007:50:350
972060994330,8cyclictest973078-21latency_hist07:30:020
97206099432,10cyclictest975639-21cat08:20:000
972071994228,10cyclictest275-21dbus-daemon11:45:013
972071994223,14cyclictest275-21dbus-daemon10:20:003
972071994221,14cyclictest37-21ksoftirqd/309:00:033
972070994234,5cyclictest974351-21kworker/2:111:45:332
972070994233,5cyclictest976433-21kworker/2:208:55:032
972060994232,6cyclictest16950irq/224-800000008:55:000
972071994128,9cyclictest37-21ksoftirqd/309:46:453
972071994128,9cyclictest3467602-21systemd-journal09:40:043
972071994128,10cyclictest37-21ksoftirqd/310:28:123
972071994128,10cyclictest37-21ksoftirqd/310:28:123
972071994125,11cyclictest37-21ksoftirqd/311:40:013
972071994124,12cyclictest1-21systemd07:30:013
97207199411,3cyclictest131rcu_preempt12:00:003
972070994132,6cyclictest971789-21kworker/2:007:45:022
972070994129,9cyclictest3467602-21systemd-journal08:20:002
972060994134,4cyclictest16950irq/224-800000009:05:020
972060994124,13cyclictest973286-21sshd07:34:020
972060994124,13cyclictest973286-21sshd07:34:020
972060994119,14cyclictest974155-21latency_hist07:50:020
972060994119,14cyclictest974155-21latency_hist07:50:020
97207199403,3cyclictest131rcu_preempt09:20:023
972071994029,8cyclictest37-21ksoftirqd/308:50:003
972071994028,9cyclictest1-21systemd12:15:013
972071994028,9cyclictest1-21systemd12:15:013
972071994028,10cyclictest37-21ksoftirqd/312:26:553
972071994028,10cyclictest37-21ksoftirqd/311:21:083
972071994026,11cyclictest37-21ksoftirqd/309:35:013
972071994026,10cyclictest37-21ksoftirqd/311:10:373
972071994025,10cyclictest37-21ksoftirqd/307:25:013
972070994032,5cyclictest971789-21kworker/2:007:30:032
972070994028,8cyclictest1-21systemd08:50:002
972060994021,9cyclictest0-21swapper/008:30:020
972060994021,9cyclictest0-21swapper/008:30:020
972071993929,7cyclictest37-21ksoftirqd/310:10:003
972071993928,8cyclictest1039187-21sshd11:19:023
972071993927,9cyclictest37-21ksoftirqd/310:04:333
972071993927,9cyclictest37-21ksoftirqd/310:04:333
972071993926,10cyclictest37-21ksoftirqd/312:32:533
972071993926,10cyclictest37-21ksoftirqd/312:32:533
972071993925,9cyclictest37-21ksoftirqd/312:40:003
972071993924,12cyclictest37-21ksoftirqd/307:15:013
972071993924,12cyclictest37-21ksoftirqd/307:15:013
972071993924,11cyclictest37-21ksoftirqd/311:32:563
97207199392,13cyclictest1012752-21ssh10:22:483
97207199391,4cyclictest0-21swapper/312:19:033
97207199391,3cyclictest131rcu_preempt09:29:313
97207199391,3cyclictest131rcu_preempt09:29:303
972070993931,5cyclictest977466-21kworker/2:309:43:092
972070993930,5cyclictest974351-21kworker/2:108:30:022
972070993930,5cyclictest974351-21kworker/2:108:30:012
97206099397,6cyclictest131rcu_preempt08:50:010
972071993828,7cyclictest37-21ksoftirqd/310:40:013
972071993828,7cyclictest37-21ksoftirqd/310:40:003
972071993828,7cyclictest37-21ksoftirqd/308:25:013
972071993827,9cyclictest37-21ksoftirqd/308:32:023
972071993827,8cyclictest37-21ksoftirqd/310:52:593
972071993827,7cyclictest275-21dbus-daemon07:50:013
972071993827,7cyclictest275-21dbus-daemon07:50:013
972071993825,10cyclictest999144-21sh09:54:033
972071993824,11cyclictest3467602-21systemd-journal08:55:003
972071993822,11cyclictest37-21ksoftirqd/310:44:593
97207199381,3cyclictest131rcu_preempt09:14:143
97207199381,12cyclictest1008470-21ssh10:13:553
972070993830,5cyclictest309-21in:imuxsock07:20:002
972070993827,8cyclictest30-21ksoftirqd/209:05:012
97206099382,32cyclictest971653-21kworker/0:007:22:390
972071993725,9cyclictest3467602-21systemd-journal07:59:023
97207199372,4cyclictest131rcu_preempt08:30:013
97207199372,4cyclictest131rcu_preempt08:30:013
972071993724,10cyclictest37-21ksoftirqd/311:05:033
972071993723,11cyclictest37-21ksoftirqd/309:10:013
972071993723,11cyclictest37-21ksoftirqd/309:10:013
972070993724,9cyclictest30-21ksoftirqd/208:31:382
972071993628,5cyclictest975638-21sendmail08:20:013
972071993626,8cyclictest37-21ksoftirqd/311:47:083
972071993625,9cyclictest37-21ksoftirqd/309:55:053
972071993623,9cyclictest37-21ksoftirqd/311:07:293
972071993623,10cyclictest37-21ksoftirqd/312:01:443
972070993624,9cyclictest30-21ksoftirqd/208:40:002
972071993525,7cyclictest977969-21sshd09:04:023
972071993522,10cyclictest37-21ksoftirqd/311:27:533
97207199351,3cyclictest131rcu_preempt07:45:023
97207199351,3cyclictest131rcu_preempt07:20:003
972070993525,6cyclictest309-21in:imuxsock08:05:012
972070993522,10cyclictest275-21dbus-daemon08:10:012
97206099351,30cyclictest0-21swapper/007:10:020
972071993427,4cyclictest361rcuc/308:15:023
97207099343,4cyclictest131rcu_preempt07:25:012
972070993424,8cyclictest30-21ksoftirqd/208:13:272
972071993325,5cyclictest1-21systemd08:45:013
972071993325,5cyclictest1-21systemd08:45:013
972071993323,6cyclictest37-21ksoftirqd/307:55:013
972071993124,5cyclictest37-21ksoftirqd/307:10:023
97207099311,3cyclictest131rcu_preempt07:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional